Research  Report  1979 


Designing  Interactive  Multimedia  Instruction 
to  Address  Soldiers’  Learning  Needs 


Paul  N.  Blankenbeckler 

Northrop  Grumman  Corporation 

Thomas  Rhett  Graves 

U.S.  Army  Research  Institute 

Richard  L.  Wampler 

Northrop  Grumman  Corporation 


United  States  Army  Research  Institute 
for  the  Behavioral  and  Social  Sciences 

December  2014 


Approved  for  public  release;  distribution  is  unlimited. 


U.S.  Army  Research  Institute 

for  the  Behavioral  and  Social  Sciences 


Department  of  the  Army 
Deputy  Chief  of  Staff,  G1 

Authorized  and  approved: 


MICHELLE  SAMS,  Ph.D. 
Director 


Research  accomplished  under  contract 
for  the  Department  of  the  Army  by 

Northrop  Grumman  Corporation 


Technical  Review  by 

Victor  Ingurgio,  U.S.  Army  Research  Institute 
Christopher  Vowels,  U.S.  Army  Research  Institute 
M.  Glenn  Cobb,  U.S.  Army  Research  Institute 


NOTICES 

DISTRIBUTION:  This  Research  Report  has  been  submitted  to  the  Defense  Information 
Technical  Center  (DTIC).  Address  correspondence  concerning  ARI  reports  to:  U.S. 
Army  Research  Institute  for  the  Behavioral  and  Social  Sciences,  Attn:  DAPE-ARI-ZXM, 
6000  6th  Street  Building  1464  /  Mail  Stop:  5610),  Fort  Belvoir,  VA  22060-5610. 

FINAL  DISPOSITION:  Destroy  his  Research  Report  when  it  is  no  longer  needed.  Do 
not  return  it  to  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences. 

NOTE:  The  findings  in  this  Research  Report  are  not  to  be  construed  as  an  official 
Department  of  the  Army  position,  unless  so  designated  by  other  authorized  documents. 


REPORT  DOCUMENTATION  PAGE 


1.  REPORT  DATE  (dd-mm-yy) 
December  2014 


2.  REPORT  TYPE 
Final 


4.  TITLE  AND  SUBTITLE 

Designing  Interactive  Multimedia  Instruction  to 
Address  Soldiers’  Learning  Needs 


3.  DATES  COVERED  (from.  .  .  to) 

May  2012  -  December  2013 


5a.  CONTRACT  OR  GRANT  NUMBER 

W5J9CQ-1 1-D-0001 


5b.  PROGRAM  ELEMENT  NUMBER 

633007 


6.  AUTHOR(S) 

Paul  N.  Blankenbeckler; 
Thomas  Rhett  Graves; 
Richard  L.  Wampler 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Northrop  Grumman  Technical  Services 

3565  Macon  Road  U.S.  Army  Research  Institute 

Columbus,  GA  31907  for  the  Behavioral  &  Social  Sciences 

6000  6th  Street 

Fort  Belvoir,  VA  22060-5610 


9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 
U.S.  Army  Research  Institute 

for  the  Behavioral  &  Social  Sciences 
6000  6th  Street  (Building  1464 /Mail  Stop  5610) 

Fort  Belvoir,  VA  22060-5610 


5c.  PROJECT  NUMBER 

A792 


5d.  TASK  NUMBER 

359 


5e.  WORK  UNIT  NUMBER 


8.  PERFORMING  ORGANIZATION  REPORT 
NUMBER 


10.  MONITOR  ACRONYM 

ARI 


11.  MONITOR  REPORT  NUMBER 
Research  Report  1979 


12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Distribution  Statement  A:  Approved  for  public  release;  distribution  is  unlimited. 


19.  LIMITATION  OF 
ABSTRACT 

20.  NUMBER 

OF  PAGES 

21.  RESPONSIBLE  PERSON 
Dorothy  Young 

16.  REPORT 

Unclassified 

17.  ABSTRACT 

Unclassified 

18.  THIS  PAGE 

Unclassified 

Unlimited 

Unclassified 

51 

703-545-2316 

l 


Research  Report  1979 


Designing  Interactive  Multimedia  Instruction  to 
Address  Soldiers’  Learning  Needs 


Paul  N.  Blankenbeckler 

Northrop  Grumman  Corporation 

Thomas  Rhett  Graves 

U.S.  Army  Research  Institute 

Richard  L.  Wampler 

Northrop  Grumman  Corporation 


Fort  Benning  Research  Unit 
Scott  E.  Graham,  Chief 


U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 

6000  6th  Street,  Building  1464 
Fort  Belvoir,  Virginia  22060 

June  2014 


Approved  for  public  release;  distribution  is  unlimited. 

ii 


ACKNOWLEDGMENT 


We  appreciate  the  collegiality  and  support  of  the  Maneuver  Center  of  Excellence  (MCoE) 
Directorate  of  Training  and  Doctrine  (DOTD),  Fort  Benning,  GA,  during  this  research. 
Specifically,  we  would  like  to  thank  Mr.  Edward  B.  Buckley  and  Ms.  Lindsey  L.  Fredman  for 
assisting  us  in  collecting  interactive  multimedia  instruction  (IMI)  source  materials  for  our 
review.  We  also  thank  Mr.  Chris  Jaques,  Mr.  John  Budd,  and  Mr.  Chuck  Taylor  for  their  interest 
and  support  of  our  project  at  the  MCoE.  Finally,  Mr.  Jay  A.  Brimstin  and  Mr.  James  M.  Morris 
deserve  special  thanks  for  identifying  the  relationship  between  point  of  need  and  interactive 
multimedia  instruction  as  a  topic  in  need  of  investigation. 

We  also  thank  the  Fires  Center  of  Excellence  (FCoE)  DOTD  for  their  interest  in  and 
support  of  this  research.  In  particular,  Mr.  Martin  L.  Huffmaster  was  essential  in  helping  us 
obtain  needed  IMI  source  materials  related  to  training  at  the  FCoE. 

For  their  assistance  in  tracking  down  IMI  materials  developed  at  Camp  Dodge,  IA,  we 
thank  COL  Gregory  O.  Haygood,  Jr.,  LTC  Peter  J.  Quinn,  CPT  Jason  T.  Huffman,  Ms.  Danielle 
Orr,  and  Mr.  Paul  Outhier  of  the  Iowa  Army  National  Guard  Distributed  Learning  Development 
Center.  Dr.  Gary  Rauchfuss  (TRADOC  Institute  for  Noncommissioned  Officer  Professional 
Development)  and  SGM  Michael  Coker  also  provided  much  needed  information  and  assistance 
in  tracking  down  IMI  for  review.  Drs.  Glenn  Cobb,  Jean  Dyer,  Peter  Schaefer,  and  Scott 
Graham  (ARI-Fort  Benning,  GA)  provided  valuable  feedback  and  recommendations  at  critical 
points  in  the  research  effort. 

Finally,  we  would  like  to  acknowledge  other  key  personnel  who  supported  this  effort. 
Designing  and  developing  IMI  required  a  multidisciplinary  team  to  provide  expertise.  These 
individuals  included  operational  researchers,  military  subject  matter  experts,  educational  and 
psychological  researchers,  computer  programmers,  and  IMI  developers.  Each  of  these 
participants  provided  insights  from  their  specific  discipline  that  contributed  to  our  final  outcome. 


DESIGNING  INTERACTIVE  MULTIMEDIA  INSTRUCTION  TO  ADDRESS  SOLDIERS’ 
LEARNING  NEEDS 

EXECUTIVE  SUMMARY 


Research  Requirement: 

The  Maneuver  Center  of  Excellence  (MCoE)  Directorate  of  Training  and  Doctrine 
(DOTD),  Fort  Benning,  GA,  requested  research  to  address  how  the  Army  Learning  Model’s 
point  of  need  concept  could  be  applied  in  the  design  and  development  of  interactive  multimedia 
instruction  (IMI).  A  first  step  in  that  process  was  to  review  and  evaluate  existing  Anny  IMI, 
which  was  the  focus  of  a  previous  report  (see  Blankenbeckler,  Graves,  &  Wampler,  2013).  The 
current  report  documents  the  process  and  rationale  for  how  we  built  upon  this  foundation  to 
design  and  develop  six  point  of  need  IMI  exemplars.  Additionally,  it  describes  key  features  of 
the  IMI  as  well  as  presents  the  exemplars.  An  experimental  test  of  the  developed  IMI  will  be 
documented  in  a  third  report. 

Procedure: 

We  documented  the  process  and  rationale  for  the  design  and  development  of  six  point  of 
need  IMI  exemplars.  The  six  IMI  exemplars  were  organized  in  terms  of  the  three  variations  of 
point  of  need  training — (a)  familiarization,  (b)  core,  and  (c)  tailored  training — and  had  as  their 
main  audience,  new  squad  or  team  leaders.  Two  topics  were  covered  by  the  IMI:  (a)  Adjust 
Indirect  Fire  and  (b)  Conduct  a  Defense  by  a  Squad.  Among  the  point  of  need  variations, 
familiarization  IMI  was  designed  to  provide  an  overview  of  a  topic,  providing  greater  breadth  of 
information  than  depth.  Core  IMI  was  designed  to  provide  in-depth  information  on  a  particular 
topic.  Tailored  training  IMI  was  designed  to  be  adaptable  to  different  learners’  needs  by 
incorporating  depth  and  breadth  of  infonnation  across  multiple  learning  paths,  and  providing 
diagnostic  pre-  and  post-tests.  In  the  tailored  training  variation,  diagnostic  test  scores  were  used 
to  provide  feedback  to  learners  and  to  help  them  select  among  alternative  learning  paths,  thereby 
personalizing  their  learning  experience. 

Findings: 

Point  of  need  IMI  is  supported  by  key  features  that  can  be  incorporated  into  the  design  of 
the  instructional  methods  and  content  to  target  different  types  of  learner  preferences  and  needs. 
First,  the  audience  must  be  identified  and  their  specific  needs  detennined  in  order  to  select 
appropriate  content  and  examples.  Concerning  the  design  of  the  IMI,  content  should  be 
organized  into  sections  that  focus  on  a  single,  narrowly  defined  topic.  Providing  links  to  the 
different  parts  of  the  IMI  upfront  helps  to  summarize  content  and  provide  learners  with  a  means 
to  navigate  through  the  IMI.  Part-task  assessment  and  feedback  can  be  used  for  sequential  tasks 
to  build  and  reinforce  the  learners’  understanding  of  tasks.  Tailored  training  techniques  can  be 
utilized  in  IMI  outside  a  learning  management  system,  using  relatively  low  cost  implementations 
in  HTML.  For  instance,  pre-  and  post-training  diagnostics  with  feedback  may  be  incorporated 
into  IMI  to  help  learners  make  self-aware  decisions  when  selecting  content  to  tailor  their  learning 
experience. 


IV 


Utilization  and  Dissemination  of  Findings: 

This  report  presents  our  findings  from  the  second  phase  of  the  Tailoring  Multimedia 
Instruction  to  Soldier  Needs  research  effort.  A  subsequent  research  report  will  describe 
experiments  with  the  IMI  exemplars  whose  development  is  presented  in  this  report.  The  results 
of  our  research  have  been  briefed  to  MCoE  DOTD,  4  November  2013,  and  have  been  presented 
as  an  information  paper  for  the  Henry  Caro  Noncommissioned  Officer  Academy  at  Fort 
Benning,  GA  in  December  2013;  and  to  the  Institute  for  Noncommissioned  Officer  Professional 
Development,  March  2014.  In  addition,  the  IMI  exemplars  presented  in  this  report  have  been 
transitioned  to  MCoE  DOTD. 
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Designing  Interactive  Multimedia  Instruction  to 
Address  Soldiers’  Learning  Needs 

Introduction 

We  developed  interactive  multimedia  instruction  (IMI)  exemplars  and  applied 
empirically  validated  instructional  design  techniques  to  address  the  Army  Learning  Model’s 
(ALM)  point  of  need  concept  (TRADOC,  2011a).  In  this  research  report,  we  describe  the 
development  process  for  six  point  of  need  IMI  exemplars  designed  to  address  two  topics  relevant 
to  new  squad  or  team  leaders:  Adjust  Indirect  Fire  and  Conduct  a  Defense  by  a  Squad.  We  also 
present  features  of  the  IMI  exemplars  we  developed,  covering  three  types  of  point  of  need 
training:  familiarization,  core,  and  tailored  training. 1 

Research  Problem 

The  Anny  has  used  interactive  multimedia  instruction  (IMI)  to  address  a  variety  of 
training  needs.  IMI  has  been  applied  as:  (a)  prerequisite  training  before  entering  courses,  (b) 
supporting  instruction  within  courses,  (c)  refresher  training,  and  as  (d)  comprehensive  instruction 
to  cover  complete  courses  (Straus,  Shanley,  Burns,  Waite,  &  Crowley,  2009;  Blankenbeckler, 
Graves,  &  Wampler,  2013).  IMI  combines  both  auditory  and  visual  media  with  interactive 
content,  designed  to  respond  to  learners’  actions.  Moreover,  IMI  is  often  delivered  at  a  distance 
using  technologies  such  as  computers,  tablets,  or  smart  phones  (Moreno  &  Mayer,  2007). 

ALM  has  presented  IMI  as  an  essential  tool  for  delivering  instruction  when,  where,  and 
how  it  is  needed;  that  is,  instruction  delivered  at  the  Soldiers’  point  of  need  (TRADOC,  2011a). 
The  point  of  need  concept  may  be  understood  in  terms  of  matching  the  right  learner  to  the  right 
infonnation  at  the  right  time  and  place.  Within  ALM,  it  is  a  concept  intended  to  guide  the  Army 
in  developing  instructional  materials  to  address  critical  learning  needs  when  they  most  need  to  be 
addressed  within  a  learner-centered  and  life-long  learning  context  (TRADOC,  2011a). 

Point  of  need  is  often  addressed  from  two  perspectives.  The  first  is  focused  on 
technologies  used  to  deliver  training.  This  includes  training  research  and  interventions  that  seek 
to  apply  technologies  across  different  learning  environments  (see  Swanson,  Ratwani,  Holland, 
Zeidman,  &  Bickley,  2013).  The  second  is  focused  on  issues  of  instructional  design,  addressing 
the  question  of  how  best  to  fit  instruction  to  learners’  identified  needs.  These  two  perspectives 
are  not  independent;  instruction  must  be  designed  within  the  possibilities  and  constraints  of  a 
selected  learning  technology  (see  Dyer,  Singh,  &  Clark,  2005;  Blankenbeckler,  Graves,  & 
Wampler,  2013). 

Even  so,  our  research  was  focused  on  the  second  perspective.  Along  with  presenting 
examples  from  the  six  IMI  exemplars,  we  will  address  various  issues  that  arose  in  our 
development  efforts  and  explain  how  we  incorporated  instructional  design  strategies  to  address 


1  A  previous  U.S.  Army  Research  Institute  (ARI)  research  report  detailed  our  evaluation  of  existing  Army  IMI  for 
transition  to  a  point  of  need  format  (see  Blankenbeckler,  Graves,  &  Wampler,  2013).  A  research  report  to  follow 
will  detail  an  empirical  evaluation  of  the  IMI  we  developed. 
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the  ALM  point  of  need  concept.  While  this  report  describes  a  specific  research  effort  to  develop 
point  of  need  IMI,  the  processes  and  problems  we  describe  may  generalize  to  other  situations  in 
which  training  developers  and  researchers  seek  to  reuse  or  develop  new  point  of  need,  tailored, 
or  adaptive  training. 


Point  of  Need  IMI  Planning  and  Design 

The  following  section  describes  our  process  for  developing  the  six  IMI  exemplars.  It  also 
reviews  critical  decision  points  and  our  rationale  for  various  development  decisions. 

Selecting  a  Platform 

The  first  decision  we  made  concerned  the  appropriate  platfonn  for  the  point  of  need  IMI 
exemplars.  We  selected  Personal  Computers  (PCs)  with  Windows  Operating  System.  While  we 
could  have  designed  for  smartphone  or  tablet  platforms,  we  chose  this  platfonn  because  of  its 
prevalence  in  the  Anny  schoolhouses.  Moreover,  given  our  focus  on  instructional  design  over 
technology,  the  PC  platfonn  would  allow  for  greater  flexibility  for  distribution  of  the  IMI 
exemplars  after  they  had  been  developed.  At  present,  smartphones  and  tablets  are  not  officially 
sanctioned  by,  nor  uniformly  present  throughout,  the  Army;  therefore,  to  design  instruction  for 
these  technologies  may  limit  accessibility  (see  Larkins,  2014).  Moreover,  most  existing  Anny 
IMI  has  been  developed  to  run  on  PCs  or  laptop  computers  as  their  primary  platfonns. 

That  said,  it  is  important  to  consider  the  role  technology  plays  in  delivering  training  at  the 
point  of  need.  The  technology  chosen  to  deliver  training  influences  its  geographic 
accessibility — i.e.,  the  training  is  available  when  and  where  it  is  needed  (Brown,  1997).  In  the 
longer  tenn,  handheld  devices  may  be  useful  for  increasing  access  to  Anny  training  materials, 
potentially  improving  Soldiers’  ability  to  reach-back  to  the  schoolhouse  when  needed.  Due  to 
their  limitations  in  screen  size,  processing  power,  etc.,  it  is  an  open  question  as  to  whether 
handheld  devices  would  be  more  or  less  effective  for  multimedia  instruction  than  PCs  or  laptop 
computers. 

When  considering  point  of  need,  it  is  possible  for  questions  about  technology  to  occupy 
too  much  focus.  Research  has  explored  whether  technology  may  help  or  hinder  applications  of 
particular  instructional  designs  and  techniques  (see  Bedwell  &  Salas,  2010;  Bower,  2008;  Clark 
&  Mayer,  2008;  Swanson  et  ah,  2013).  In  fact,  some  researchers  have  claimed  “technology 
matters  only  to  the  extent  that  it  may  allow  new  instructional  methods”  (Mayer  interview  in 
Veronikas  &  Shaughnessy,  2005).  A  focus  on  the  latest  technology  without  also  exploring  how 
it  impacts  instructional  design  will  not  likely  accomplish  what  ALM  envisions.  There  is  no 
sense  in  promoting  technology  without  considering  how  to  educate  and  train  effectively  with  that 
technology  (Bedwell  &  Salas,  2010). 


2  Complete  copies  of  the  six  IMI  exemplars  are  available  from  Dr.  Rhett  Graves  at  the  U.S.  Army  Research  Institute 
on  request:  thomas.r.graves5.civ@mail.mil.  The  IMI  exemplars  cover  two  topic  areas  relevant  to  new  Combat 
Arms  squad  or  team  leaders  (i.e.,  Specialist  or  Corporal  [E-4]  and  Sergeant  [E-5]  levels):  Adjust  Indirect  Fire  and 
Conduct  a  Defense  by  a  Squad. 
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Moreover,  if  one  takes  training  designed  for  one  technology  and  attempts  to  access  it  on 
another  platform,  the  transition  can  lead  to  problems.  The  design  and  format  of  legacy  training 
may  be  ill-suited  to  newer  technologies.  Consider,  for  example,  the  potential  problems  arising 
from  disparate  screen  sizes  when  training  intended  for  the  PC  is  accessed  through  a  web-browser 
on  a  handheld  device.  Moving  between  technologies  without  making  appropriate  adaptations  to 
the  design  and  format  of  the  media  could  ultimately  undennine  learning  (see  Moreno  &  Mayer, 
2007;  Clark  &  Mayer,  2008). 

Three  Types  of  Point  of  Need  IMI 

Although  the  ALM  has  not  specified  the  particular  types  of  needs  learners  may  have,  we 
focused  on  the  following  in  order  to  cover  a  broad  range  of  potential  learning  needs: 
familiarization,  core,  and  tailored  training  (see  Blankenbeckler,  Graves,  &  Wampler,  2013). 
Familiarization  IMI  was  intended  to  provide  learners  with  an  overview  of  the  topic  and  resources 
to  learn  more,  i.e.,  breadth  of  information.  It  would  assist  learners  in  gaining  an  overview  of  the 
topic  domain,  but  not  a  high-level  of  proficiency.  Familiarization  IMI  was  further  intended  for 
learners  who  may  need  to  be  prepared  for  more  in  depth  learning,  assisting  them  in  forming  a 
basic  understanding  of  the  content  domain. 

Core  IMI  was  intended  to  provide  only  the  essential  infonnation  learners  would  need  to 
perform  a  narrowly  defined  task,  i.e.,  it  provided  depth  of  information,  with  less  breadth  than 
familiarization  IMI.  When  designing  core  IMI,  we  thought  in  terms  of  ‘how-to’  manuals  that 
present  a  step-by-step  approach  to  task  execution.  Again,  given  that  the  focus  was  on  point  of 
need,  we  did  not  intend  the  learner  to  develop  a  high-level  of  proficiency.  Core  IMI  was 
designed  for  learners  who  would  need  immediate  support  in  conducting  on-the-job  tasks  or  a 
quick  refresher  of  previously  learned  skills.  This  type  of  training  would  not  lead  to  mastery,  as 
the  core  IMI  contained  no  hands-on  perfonnance  or  sophisticated  simulation. 

Finally,  tailored  training  IMI  combined  both  breadth  and  depth  of  information  into 
multiple  user-selected  learning  paths.  It  also  included  pre-  and  post- training  diagnostic 
assessments  with  individualized  feedback  following  each  assessment.  Feedback  to  learners  was 
based  on  the  results  of  their  pre-training  diagnostic  assessment.  The  feedback  was  intended  to 
help  learners  identify  content  areas  in  which  they  were  deficient  and  to  select  an  appropriate 
learning  path.  After  learners  completed  training,  they  were  able  to  take  the  post-training 
diagnostic  assessment  and  receive  feedback,  with  an  option  for  supplementary  training.  The 
tailored  training  IMI  design  was  intended  to  meet  a  variety  of  learning  needs.  The  diagnostic 
assessments  help  learners  to  be  more  aware  of  their  specific  learning  needs  and  to  make 
reasonable  decisions  on  how  to  tailor  their  learning  experience  to  meet  those  needs.  While  our 
approach  to  tailored  training  did  not  make  use  of  cutting-edge  technologies  (such  as  computer- 
adaptive  testing),  it  represents  an  effective  and  low-cost  solution  to  enable  tailored  training  in 
Anny  IMI. 


3 


Selecting  Topics  for  the  Exemplars 


We  considered  input  from  a  variety  of  sources  when  selecting  the  topics  for  our  exemplar 
IMI.  The  Maneuver  Center  of  Excellence  (MCoE),  Directorate  of  Training  and  Doctrine 
(DOTD)  had  expressed  interest  in  new  IMI  relevant  to  new  Squad  or  Team  Leaders  (i.e., 
Specialist  or  Corporal  [E-4]  and  Sergeant  [E-5]  levels).  We  also  considered  how  much  material 
we  would  be  able  to  reuse  when  developing  the  point  of  need  exemplars.  We  selected  two  topics 
to  support  the  interests  of  MCoE  DOTD  and  the  point  of  need  requirements:  Adjust  Indirect 
Fire,  and  Conduct  a  Defense  by  a  Squad.  Table  1  provides  an  overview  of  the  point  of  need  IMI 
variations  by  their  topics,  types,  and  titles. 


Table  1 

Six  Point  of  Need  IMI  Exemplars 


Topic 

Type  of  IMI 

Title 

Adjust  Indirect  Fire 
(Task  #061-283-6003) 

Familiarization 

Engaging  Targets  with  Supporting  Fires 

Core 

Conduct  Immediate  Suppression 

Tailored  Training 

Adjust  Indirect  Fire 

Conduct  a  Defense  by  a 
Squad 

(Task  #071-430-0002) 

Familiarization 

Prepare  Positions  for  Crew-Served  Weapons 

During  an  Urban  Operation 

Core 

Designate  and  Prepare  Urban  Fighting  Positions  for 
a  Javelin  Team 

Tailored  Training 

Conduct  a  Defense  by  a  Squad  in  an  Urban 

Operation 

The  tasks,  conditions,  and  standards  for  the  selected  topics,  as  documented  in  the 
Soldier’s  Manual  of  Common  Tasks,  Warrior  Leader  Skills  Level  2,  3,  and  4  (U.S.  Anny,  2008), 
are  summarized  in  Table  2. 
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Table  2 

Summary  of  Selected  Tasks,  with  Conditions  and  Standards 


Task 

Conditions 

Standards 

Adjust  Indirect  Fire 
(Task#  061-283-6003) 

Given  a  pair  of  binoculars,  a  radio, 
a  compass,  pencils,  a  coordinate 
scale,  a  map  of  the  target  area,  a 
target  to  engage  within  the  area, 
and  grid  location  of  friendly 
troops. 

Determine  the  target  location  to  within  250 
meters  of  its  actual  location.  Transmit  the 
initial  call  for  fire  within  3  minutes  after 
identifying  the  target.  Send  adjustments 
within  45  seconds  after  each  round  impacts. 
Enter  the  fire-for-effect  phase  using  no  more 
than  six  rounds  (initial  round  plus  five  for 
adjustment).  Fire  for  effect  within  50  meters 
of  the  target  using  successive  bracketing 
procedures  (or  creeping  fire  if  danger  close 
by). 

Conduct  a  Defense 
by  a  Squad 
(Task  #071-430-0002) 

Given  a  squad,  a  priority  of  work, 
and  locations  for  crew-served 
weapons  designated  by  the  platoon 
leader. 

Accomplish  preparation  of  a  defensive 
position  within  the  time  specified  in  the 
platoon  leader’s  order  while  maintaining 
security,  camouflage,  and  concealment. 

1.  Designate  fighting  positions  for  squad 
members. 

2.  Designate  alternate  and  supplementary 
positions  for  squad  members. 

3.  Ensure  assigned  priority  of  work  is 
followed  by  all  squad  members. 

4.  Maintain  security. 

5.  Continue  work  as  rapidly  as  possible. 

6.  Maintain  camouflage  and  concealment  (to 
include  noise,  light,  and  litter  discipline). 

7.  Construct  positions  properly. 


Both  tasks  described  in  Table  2  focus  on  essential  infantry  fire  team  and  squad  leader 
skills.  However,  neither  task  is  unique  to  the  infantry;  these  are  Common  Skill  Warrior  Leader 
Tasks  (U.S.  Anny,  2008).  Therefore,  they  are  applicable  to  leaders  who  must  prepare,  organize, 
or  execute  combat  operations  in  addition  to  their  unit’s  primary  mission.  Given  that  infantry 
units  may  be  assigned  missions  to  conduct  defensive  operations  as  part  of  unit  security  and  force 
protection,  we  concluded  that  the  Conduct  a  Defense  by  a  Squad  topic  was  particularly  relevant. 
Likewise,  the  Adjust  Indirect  Fire  task  is  also  applicable  to  a  broad  range  of  combat  operations. 
Leaders  in  non-maneuver  units  may  need  to  employ  indirect  fires  to  protect  themselves  or  to 
engage  a  target  of  opportunity.  Infantry  units  must  be  able  to  integrate  indirect  fires  and  employ 
combined  arms  effects  in  all  aspects  of  offensive  and  defensive  operations. 
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Adjust  Indirect  Fire  is  a  Skill  Level  2  task  focused  on  the  Sergeant  (SGT;  E-5)  fire  team 
leader. 3  The  task  has  received  more  attention  since  Anny  Warrior  Training  Plan  required  it  to 
be  introduced  and  trained  in  the  Warrior  Leader  Course  (WLC),  and  then  sustained  annually  in 
the  unit.  While  Fire  Support  Teams  (FISTs)  attached  to  maneuver  companies  or  platoons  have 
among  their  primary  duties  fire  support  operations,  fire  planning  and  target  engagement,  the 
small  unit  leader  may  be  required  to  call  for  and  adjust  indirect  fire  without  the  assistance  from  a 
FIST.  This  contingency  can  arise  due  to  their  mission  or  their  location  on  the  battlefield.  Adjust 
Indirect  Fire  is  an  individual  task,  performed  by  one  Soldier. 

The  Conduct  a  Defense  by  a  Squad  task  is  Skill  Level  3,  focused  on  the  Staff  Sergeant 
(SSG),  squad  or  section  leader;  it  is  also  applicable  to  the  Corporal,  Sergeant,  or  fire  team  leader. 
The  Army  Warrior  Training  Plan  indicates  that  this  task  will  be  trained  initially  in  the  unit  and 
then  sustained  annually.  Leaders  develop  and  implement  defense  and  force  protection,  as  well  as 
provide  plans  for  security.  These  plans  and  fires  must  be  coordinated  with  adjacent  units. 
Moreover,  leaders  who  are  preparing  and  executing  these  plans  must  often  oversee  subordinates 
to  coordinate  the  various  tasks  involved.  A  plan  must  be  developed  to  position  units  or 
individual  weapons  or  teams,  improve  positions,  integrate  fires,  and  assure  depth  and  mutual 
support.  Of  note,  an  essential  condition  of  this  task  is  the  availability  of  a  squad  of  appropriately 
armed  and  organized  Soldiers.  These  Soldiers  must  occupy  positions  selected  or  designated  for 
them,  have  these  positions  and  fields  of  fire  verified  or  adjusted  by  leaders,  improve  the  positions 
for  survivability  and  lethality  (by  improving  fields  of  fire),  and  prepare  to  execute  mutually 
supporting  fires  and  actions  to  destroy  the  enemy  or  repel  an  attack. 

We  also  selected  these  two  tasks  because  of  differences  in  task  attributes  and  perfonnance. 
For  example,  Adjust  Indirect  Fire  is  a  sequential  task,  with  a  standard  procedure  for  actions 
taken  to  request  indirect  fires  on  a  target.  Executing  that  procedure  also  requires  some 
proficiency  in  map  reading.  Radio  calls  for  fire  are  normally  communicated  in  three 
transmissions  with  standard  infonnation  in  each  transmission.  When  a  target  is  not  located 
precisely,  the  impact  of  rounds  is  corrected  using  deviation  corrections  to  the  observer-target  line 
and  the  bracketing  method  for  range  adjustments.  Bracketing  is  normally  accomplished  by 
successively  firing  over  (beyond)  or  short  of  the  target,  reducing  in  a  methodical  manner  the 
distance  between  subsequent  round  impacts  until  the  threshold  for  “fire-for-effecf  ’  (normally 
rounds  impacting  within  +/-  50  meters  of  the  target)  is  obtained.  After  “fire-for-effecf  ’  the 
observer  reports  the  effects  on  the  target,  continues  adjustment  or  provides  target  refinement 
infonnation,  and  then  ends  the  mission.  While  there  are  variations  to  this  sequence  based  on  the 
enemy  target,  desired  mission  type,  or  proximity  to  friendly  troops,  most  missions  requested  by  a 
small  unit  combat  leader  follow  this  sequence. 


3  See  Soldier’s  Manual  of  Common  Tasks,  Warrior  Leader  Skills  Level  2,  3,  4  (STP  21-24-SMCT),  September 
2008.  Enlisted  MOSs  in  the  Army  have  associated  skill  levels.  These  skill  levels  relate  to  a  Soldier's  paygrade  as 
well  as  the  expected  difficulty  of  duties  or  tasks  to  be  performed,  and  the  range  of  managerial  and  supervisory  duties 
to  be  accomplished.  For  example.  Skill  Level  1  is  an  entry-level  position  requiring  the  performance  of  tasks  under 
direct  supervision.  This  skill  level  consists  of  all  Soldiers  in  the  ranks  of  Private  (E-l)  up  to  Specialist  or  Corporal 
(E-4).  Skill  Level  2  is  a  position  requiring  the  performance  of  more  difficult  tasks  under  general  supervision. 
Soldiers  reach  skill  level  2  when  they  are  promoted  to  Sergeant  (E-5).  In  many  instances,  a  Sergeant  will  supervise 
Skill  Level  1  Soldiers. 


6 


In  contrast,  Conduct  a  Defense  by  a  Squad  tends  to  be  more  cognitive  than  procedural.  It 
involves  selectively  perceiving  relevant  aspects  of  the  battlefield  context,  understanding  the 
assigned  mission(s),  and  being  able  to  apply  basic  fire  control  principles  in  that  context.  For 
example,  the  leader  prepares  and  plans  for  the  task  using  Troop  Leading  Procedures  (TLP)  and 
Principles  of  Fire  Control.  The  steps  of  TLP  are  not  rigidly  sequential.  Some  steps  may  be 
omitted  or  overlapped  with  other  steps.  In  support  of  TLP,  mission  analysis  is  guided  by  factors 
such  as  mission,  enemy,  weather,  terrain,  availability  of  troops  and  support,  time,  and  other  civil 
considerations.  No  two  missions  are  alike.  So,  even  though  the  process  for  establishing  primary, 
alternate,  and  supplementary  positions  for  units  may  have  similar  characteristics,  the  specifics  of 
TLP,  Fire  Control  Measures,  and  other  position  preparation  considerations  are  needed  for  the 
defense  task.  The  leader  must  know  the  capabilities  of  the  squad’s  organic  and  supporting 
weapons,  apply  principles  of  fire  control,  apply  reasonable  tactical  judgments,  execute  assigned 
occupation  and  preparation  tasks  in  priority,  and  employ  appropriate  aspects  of  mission 
command  in  planning,  preparing,  executing,  consolidating,  and  recovering  from  the  mission. 

Another  reason  we  selected  these  two  tasks  was  based  on  the  availability  of  viable  IMI 
source  materials  that  we  could  reuse  in  developing  the  exemplars.  Information  pertaining  to  the 
material  selected  for  reuse  is  provided  in  Table  3. 


Table  3 

Selected  Tasks  and  Existing  IMI  Relationships* 


Base  Task 

Existing  IMI 

Proponent 

Source  for  Files  and 
Materials 

Delivery  Media 
for  Existing  IMI 

061-283-6003 

061-C01-1067: 

Field 

FCoE  DOTD 

CD-ROM;  AKO 

Adjust  Indirect 

Conduct  an 

Artillery 

via  the  My 

Fire 

Immediate 

School, 

Training  Tab  Site, 

Suppression 

FCoE 

061  Field  Artillery 

Mission  (with 

School 

embedded 

Knowledge 

enabling  Skills 

Center,  and  over 

and  Knowledge  - 

the  web  from  the 

Call  for  Fire  and 

FCoE  Reach 

Subsequent 

Back/Reset 

Corrections) 

Training 

071-430-0002 

071-326-0550: 

Infantry 

Iowa  Army  National 

AKO  via  the  My 

Conduct  a  Defense 

Prepare  Positions 

School, 

Guard  Distributed 

Training  Tab  Site, 

by  a  Squad  with 

for  Individual 

MCoE 

Learning  Development 

071  Infantry 

added  conditions: 

and  Crew-Served 

Center 

School 

Urban  operations; 

Weapons  During 

Knowledge  Center 

position/reposition 

an  Urban 

/control  crew- 
served  weapons 
(Javelin  and  M240B 
Machine  Gun) 

Operation 

*Note:  FCoE  is  Fires  Center  of  Excellence;  MCoE  is  Maneuver  Center  of  Excellence;  AKO  is  Army  Knowledge 
Online. 
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IMI  Design  Process 


Army  guidance  outlines  an  extensive  planning  and  development  process  required  for 
most  IMI  projects  (see  U.S.  Army  Training  and  Doctrine  Command  Pamphlet,  TRADOC  Pam 
350-70-2;  TRADOC,  2003).  We  used  this  basic  process  to  guide  our  development  efforts, 
adapting  the  process  as  necessary  to  incorporate  existing  IMI  and  to  work  around  various 
problems  we  encountered.  In  selecting  topics  and  operationally  defining  point  of  need,  we 
considered  MCoE  DOTD’s  interest  in  new  squad  and  team  leaders.  In  fact,  clearly  defining  the 
intended  audience  was  essential  to  determining  the  needs  we  were  going  to  address  with  the 
point  of  need  IMI.  Defining  the  intended  audience  at  the  outset  helped  to  ensure  IMI  design  and 
content  were  appropriate. 

Point  of  need  training  requires  specificity  (Blankenbeckler,  Graves,  &  Wampler,  2013). 

A  point  of  need  design  seeks  to  identify  and  meet  specific  learning  needs.  It  does  so  by  focusing 
on  the  learning  needs  of  an  identified  group  (i.e.,  new  squad  or  team  leaders)  and  the  unique 
background  knowledge  and  experience  of  individual  learners  within  that  group.  Past  IMI  seems 
to  have  been  developed  to  meet  the  training  needs  of  the  largest  possible  audience.  When  the 
audience  is  defined  too  broadly,  some  learners  may  find  IMI  contains  too  much  or  too  little 
information  and  others  may  find  that  the  IMI  content  is  confusing  or  uninteresting.  Point  of  need 
IMI  is  designed  to  target  specific  individual  and  group  learning  needs  in  order  to  keep  the 
training  as  succinct  and  focused  as  possible  for  each  learner. 

The  target  audience  was  identified  as  future  fire  team  leaders  and  squad  leaders.  On  this 
basis,  we  detennined  that  the  training  could  build  on  experiences  and  knowledge  the  audience 
would  have  gained  from  deployments,  in-unit  training,  and  home  station  training.  Although 
much  reset  training  over  the  past  12  years  has  focused  on  Southwest  Asian  missions,  this  training 
is  now  emphasizing  decisive  operations  in  a  high  multispectrum  threat  environment  involving 
maneuver  and  combined  anns  integration.  On  this  basis,  we  detennined  that  the  IMI  should 
depart  from  counterinsurgency  and  security  operations,  and  instead  deal  with  sophisticated 
mechanized  threats  and  enemy  forces  with  parity  in  equipment  capabilities  and  combat  potential 
(see  ADRP  3-0,  Department  of  Anny,  2012). 

Reusing  Parts  of  Existing  IMI 

We  sought  to  reuse  parts  of  existing  IMI  to  evaluate  whether  reuse  is  a  plausible  strategy 
to  address  the  ALM  point  of  need  concept,  transfonning  existing  Army  IMI  to  fit  the  new  model. 
Despite  being  SCORM  compliant,  the  existing  IMI  proved  to  be  challenging  to  reuse,  a  finding 
mirrored  in  similar  efforts  (see  Shanley  et  ah,  2009).  In  fact,  Shanley  et  al.  (2009)  found  that  for 
most  organizations,  significant  returns  from  IMI  reuse  were  exceptions.  Most  organizations  in 
the  Shanley  et  al.  study  reported  significant  technical  issues  and  unexpected  problems  arising  in 
their  IMI  reuse  efforts. 

Previously  we  reviewed  Anny  Combat  Anns  IMI  to  determine  if  existing  source  files 
(graphics,  images,  programming,  etc.)  could  be  adapted  to  a  point  of  need  training  format  (see 
Blankenbeckler,  Graves,  &  Wampler,  2013).  We  concluded  that  the  approach  is  possible  and  the 
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Army  would  likely  invest  more  time  and  resources  to  reuse  existing  IMI  than  to  develop  new 
IMI.  All  the  IMI  we  reviewed  needed  new  content  to  address  the  point  of  need  variations  we 
were  targeting.  It  was  not  a  plausible  strategy  to  simply  redesign  and  repackage  existing  content. 
As  an  estimate,  new  content  accounted  for  about  70%  of  the  total  content  used  in  our  IMI 
exemplars  for  this  study.  Only  about  30%  of  what  we  developed  was  able  to  use  unmodified 
content  of  existing  IMI. 

For  example,  the  Adjust  Indirect  Fire  familiarization  IMI  lesson  adapted  elements  and 
extracted  pieces  from  existing  IMI,  FCoE  06 1-C0 1-1067  Conduct  of  an  Immediate  Suppression 
Mission.  The  elements  selected  for  reuse  came  from  three  topics  within  the  existing  FCoE 
module.  Minor  changes  to  narration  and  on  screen  text  were  required  to  meet  doctrinal  accuracy. 
Some  outdated  images  were  replaced  with  action  images  showing  Soldiers  in  current  Army 
Combat  Uniform.  All  materials  unique  to  digital  calls  for  fire  were  deleted  to  better  conform  to 
the  conditions  of  the  common  skill  task.  Much  of  the  existing  material  was  developed  using 
VBS2  video.4  With  few  exceptions,  these  videos  and  animated  segments  were  maintained  and 
integrated  into  the  training.  This  lesson  provided  the  highest  reuse  of  existing  materials  with  28 
of  the  29  storyboard  frames  being  derived  from  existing  IMI.  Only  limited  image  replacement 
and  on  screen  text  corrections  were  required  and  these  were  simplified  by  the  availability  of 
source  files  for  the  original  materials. 

Similarly  the  familiarization  IMI  lesson  for  squad  defense  reused  portions  of  existing  IMI 
from  MCoE  071-326-0550  Prepare  Positions  for  Individual  and  Crew-Served  Weapons  During 
an  Urban  Operation.  Only  two  topics  from  the  existing  lesson  were  incorporated  in  their  totality. 
These  existing  materials  adequately  addressed  what  to  do  to  prepare  a  “selected”  crew-served 
weapon  position.  Additionally,  1 1  existing  images  were  drawn  from  the  lesson  and  reused  in  the 
newly  developed  IMI.  However,  the  logical  foundation  and  underlying  rationale  that  leaders 
need  to  make  position  selection  decisions  for  crew-served  weapons  could  not  be  located  in  any 
existing  IMI.  Topics  focusing  on  training  leaders  (e.g.,  determine  fields  of  fire  requirements  and 
implement  fire  control)  were  missing  from  this  lesson.  Therefore,  the  topic,  Selecting  Weapon 
Positions  and  Controlling  Fires,  was  included  in  the  new  IMI.  While  we  had  to  develop  new  IMI 
for  the  second  topic,  Common  Preparation  Considerations,  we  made  extensive  use  of  existing 
imagery  as  well. 

For  the  core  IMI  lesson  for  Adjust  Indirect  Fire,  we  adapted  the  tutorial  from  the  lesson, 
Conduct  of  an  Immediate  Suppression  Mission.  However,  to  meet  the  instructional  need  of 
addressing  “step-by-step  task  execution,”  revisions  were  required  to  the  existing  materials  for 
doctrinal  accuracy,  including  narration,  images,  and  internal  navigation.  These  revisions 
included: 

•  A  more  complete  learning  objective  and  explanation  of  the  purpose  of  the  mission  type. 

•  Addition  of  doctrinally  correct  map  graphics  to  supplement  the  threat  aspect  of  the 
scenario. 

•  Additional  materials  to  explain  map  grid  accuracies  and  required  elements  of  the  call  for 
fire  transmission. 


4  VBS2  is  the  Virtual  Battle  Simulation  Software  manufactured  and  copyrighted  by  Bohemia  Interactive 
Simulations.  It  is  used  by  the  Army  as  a  tool  to  simulate  various  tactical  scenarios. 
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•  Supplemental  infonnation  on  target  description  to  enhance  initial  effects  on  target. 

•  Supplemental  infonnation  on  “direction  to  target”  data,  computing  the  “observer  target” 
distance,  as  well  as  obtaining  and  converting  mil  angle  measurements  should  adjustments 
be  required. 

•  Development  of  a  review  of  the  core  knowledge  and  skills  for  executing  the  mission. 

The  core  IMI  lesson  for  defensive  skills  reused  some  images  from  existing  materials. 
However,  other  than  adapting  the  eight  images  from  existing  materials,  the  73  pages  of  the 
lesson,  Designate  and  Prepare  Urban  Fighting  Positions  for  a  Javelin  Team,  were  developed 
anew. 


The  comprehensive  tailored  training  IMI  lessons  proved  the  most  challenging  to  develop. 
New  developmental  materials  were  required  for  both  the  assessments  and  the  comprehensive 
training.  The  content  was  framed  in  tactical  situations  to  facilitate  a  problem-centered  approach, 
helping  learners  to  increase  their  understanding  through  solving  concrete  problems  (see  Hmelo- 
Silver,  2004).  Some  existing  familiarization  and  core  materials  were  integrated  into  the  tailored 
training  IMI.  Of  note,  these  topics  focused  on  narrow  knowledge  or  skills  sets,  identified  as 
deficiencies  through  the  pre-  or  post-training  assessments.  The  pre-  and  post-assessments  were 
also  problem  centered  and  were  developed  to  address  each  step  and  sub-element  of  the  particular 
task.  While  the  media  imposed  some  constraints  to  examining  the  full  task,  efforts  were  focused 
on  replicating  what  the  Soldier  would  observe,  experience,  or  perform  within  media  capabilities, 
to  provide  a  rigorous  part-task  assessment.  All  assessments  were  conducted  in  the  context  of 
solving  a  real-world  problem  in  a  tactical  scenario.  For  example,  in  the  Adjust  Indirect  Fire 
tailored  training  IMI,  the  user  was  required  to  initiate  indirect  fires  against  and  adjust  onto  a 
target  of  opportunity.  Figure  1  provides  a  typical  assessment  storyboard  illustrating  features  of 
the  assessment  design  and  assessment  of  skills  required  for  one  of  the  task  steps.  While  the  user 
was  provided  multiple  choice  responses  to  select  from,  there  were  no  easy  exclusions.  In  the 
Adjust  Indirect  Fire  IMI,  the  user  selection  normally  provided  only  one  correct  response,  while 
the  Conduct  a  Defense  by  a  Squad  tailored  training  IMI  provided  multiple  correct  answers. 

We  experienced  various  constraints  when  repurposing  existing  IMI,  including:  (a) 
unavailable  or  incomplete  source  files,  (b)  incompatibility  of  archived  files  with  current 
software,  (c)  outdated  doctrine  and  graphics,  (d)  revisions  to  computer  code  propagating 
unintended  errors  in  the  existing  programming,  and  (e)  difficulties  separating  the  existing  IMI 
from  its  interface  with  a  learning  management  system  (LMS).  Each  of  these  constraints  could 
make  any  development  effort  very  costly  with  regard  to  time  and  other  resources,  thereby 
limiting  any  effort  to  transfonn  or  repackage  existing  IMI  for  point  of  need.  However,  we 
developed  some  workaround  solutions  to  address  these  issues.  For  instance,  segments  of  video 
and  audio  can  be  recorded  from  existing  IMI  training  without  any  major  loss  of  resolution  (i.e., 
digital  to  digital)  if  the  source  code  is  not  available  or  if  modifications  at  the  level  of  source  code 
are  not  required.  These  sampled  portions  of  existing  IMI  may  then  be  incorporated  as  piece- 
parts  in  a  new  IMI  training  module.  While  this  is  clearly  not  an  optimal  solution,  it  is  a  field- 
expedient  solution  that  is  inexpensive  and  effective. 
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Typical  Assessment  Storyboard 


-.*•**  r*  c»'  **•*•■»  i :i 

Direction  and  distance 
may  be  used  to  determine 
target  location,  but  those 
factors  are  essential  when 
making  adjustments. 

What  is  the  direction  and 
estimated  distance  to  this 
target  from  your  OP? 


Narration:  While  not  always  an  element  of  the  initial  call  for  fire  transmission,  the 
direction  and  distance  to  target  may  become  essential  to  the  success  of  the  mission  if 
adjustments  are  required.  Use  the  lensatic  compass  and  binocularreticle  image  to 
assist  you.  Determine  and  select  the  approximate  direction  and  distance  to  the  target 
from  your  squad's  observation  post. 


1.  Identified  as  new  material 
vs.  reused  existing  material 

2.  Assessments  are  framed  in 
the  context  of  problem  solving 
in  a  tactical  situation. 

3.  All  assessment  questions  call 
for  the  performance  of  (whole  or 
part  task)  skills  in  the  context  of 
the  tactical  situation.  This 
example  requires  map  reading, 
use  of  a  lensatic  compass,  use  of 
the  binocular  reticle,  and  the  mil 
relation  (WORM)  formula. 

4.  Incorrect  response 
selections  were  developed 
to  include  common 
mistakes;  there  were 
seldom  “throw  away”  or 
easy  discriminators  in 
selections. 


Figure  1.  A  typical  assessment  storyboard  annotated  to  highlight  some  of  the  instructional 
design  features. 

The  difficulties  involved  in  reusing  existing  media  are  significant  in  that  the  Sharable 
Content  Object  Reference  Model  (SCORM)  was  intended  to  make  training  materials  portable, 
durable,  and  reusable  (DoD,  2011). 5  In  many  cases,  problems  arise  that  interfere  with  the 
reusability  of  existing  content.  The  components,  identified  as  Sharable  Content  Objects  (SCOs), 
reside  in  the  media  and  in  the  source  files  used  to  construct  the  media.  We  were  only  able  to 
reuse  SCOs  when  they  were  available  or  extractable  from  the  existing  media.  Extracting 
material  (i.e.  graphics,  video,  narrations,  etc.)  means  repurposing  its  current  use  as  existing 
programmed  media  or  obtaining  it  through  existing  source  files. 

Source  files  were  not  always  available.  As  course  proponents  have  shifted,  responsible 
officials  and  staffing  have  changed.  Archival  files  and  records,  including  source  files,  are  not 
always  transferred,  retained,  or  maintained.  If  retained,  they  may  no  longer  be  available  in  a 
form  that  permits  retrieval,  modification,  or  further  sharing.  These  are  significant  problems  that 
undermine  the  Army’s  efforts  to  operate  in  accord  with  the  SCORM  approach. 


5  SCORM  is  a  standard  for  a  technical  framework  to  enable  the  use  of  Web-based  e-Learning  content  across 
multiple  environments  (e.g.,  LMSs).  SCORM  defines  how  individual  instruction  elements  are  combined  at  a 
technical  level  and  sets  conditions  for  the  software  needed  to  use  the  content.  For  further  explanation,  see  “SCORM 
2004,  3rd  Edition”  (DoD,  2008). 
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When  source  files  were  available,  the  legacy  software  used  to  create  those  files  was 
frequently  not  compatible  with  current  versions,  necessitating  various  workaround  solutions. 
Most  commercial  software  companies  strive  to  maintain  some  backward  compatibility  in 
successive  releases  of  their  software.  However,  the  market  segment  for  these  companies  is  very 
competitive,  so  they  have  strong  incentive  to  innovate  new  features  and  capabilities.  The  version 
of  software  used  to  program  an  existing  IMI  module  may  no  longer  be  identifiable,  technically 
compatible  with  newer  operating  systems,  available  on  the  market,  or  supported  in  current 
versions  of  the  software. 

Similar  compatibility  problems  can  arise  when  existing  IMI  is  extracted  from  an  LMS.6 
An  LMS  is  a  software  application  designed  for  the  administration,  documentation,  tracking, 
reporting,  and  delivery  of  e-leaming  courses  and  training  programs.  It  is  a  critical  element  in  the 
mass  distribution  of  e-learning  courseware  and  defines  the  communications  between  the  client  or 
learner  and  the  host  system.  Within  the  IMI,  an  LMS  defines  the  learning  path  for  the  learner 
and  assures  their  compliance  with  course  requirements.  When  course  materials  are  extracted 
from  the  LMS,  it  is  important  to  keep  in  mind  the  points  at  which  the  host  LMS  and  the  hosted 
media  intertwine  may  act  like  a  software  virus,  blocking  or  masking  desired  functions.  Finding 
these  intersecting  points  in  the  source  code  and  removing  them  can  be  very  challenging,  since  no 
two  LMSs  work  exactly  the  same  way.  Moving  materials  from  one  LMS  to  be  reused  in  another 
may  introduce  a  number  of  technical  challenges  that  often  can  only  be  resolved  by  an  expert 
programmer. 

Other  problems  can  arise  once  source  content  is  modified  for  reuse.  Revisions  to  the 
programmed  instructional  media  made  during  development  may  employ  techniques  to  obscure  or 
mask  undesired  functions  or  content.  When  revisions  are  handled  in  this  way,  the  error  remains 
packaged  in  the  media  but  is  now  obscured  by  new  material,  programming  techniques,  or  has 
been  adjusted  to  display  and  run  in  the  desired  manner.  When  working  with  the  source  files  to 
extract  a  section  of  IMI  for  reuse,  these  hidden  problems  can  interfere  with  the  process.  Given 
these  issues,  developing  IMI  from  anew  may  be  the  better  alternative  in  terms  of  time  and  cost. 

Storyboard  Development 

Our  storyboard  development  process  was  intended  to  accommodate  both  reused  and 
newly  developed  IMI  materials.  While  new  material  could  be  developed  to  accord  with  our 
selected  pedagogical  approach,  the  existing  materials  we  selected  would  also  need  to  mirror  that 
approach.  While  our  desire  was  that  existing  materials  could  be  used  as  is,  most  had  to  be 
revised  to  ensure  continuity  between  our  pedagogy  and  instructional  design.  Further,  we 
recognized  that  we  needed  to  ensure  that  correct  doctrine  was  being  presented  in  order  to  meet 
desired  learning  outcomes.  A  number  of  images  and  narrations  from  existing  IMI  had  to  be 
revised  to  be  accurate  and  current,  especially  in  regards  to  reflecting  current  unifonn 
combinations  and  equipment.  It  was  critical  to  avoid  any  compromises  concerning  tactics  and 
doctrine.  If  these  conditions  could  not  be  met,  the  materials  were  not  reused. 


6  LMSs  employed  by  the  Armed  Forces  must  meet  high  standards  and  specifications  for  web-based  e-learning  and 
must  be  SCORM  compliant. 


12 


Instructional  designers  used  a  storyboard  format  common  to  the  traditional  IMI 
development  process.  In  particular,  Microsoft’s  Office  2007  PowerPoint  note  pages  were  easily 
adaptable  to  this  task.  The  note  pages  served  as  a  workbench  for  assembling  the  required 
elements  the  designer,  reviewer,  and  programmer  needed  to  approve  and  build  the  IMI.  While 
programming  may  combine  the  elements  of  multiple  storyboards,  a  storyboard  represents  a 
single  unit  of  the  media  and  should  provide: 

•  Administrative  reference  information  -  file  name,  page  number,  edition,  as  well  as 
information  to  identify  key  members  of  the  production  staff  or  principles. 

•  On  screen  text. 

•  Visual  media  display  data  -  graphics,  photographs,  tables,  animation,  and  video 
segments. 

•  On-screen  treatments,  programming,  triggers,  controls,  and  user  interface  details  for 
actions  on  the  page. 

•  Primary  navigation  information  for  forward  progression  or  returning  to  previous  pages. 

•  Secondary  navigation  and  action  information  for  automatic,  conditional,  or  user  selected 
options  such  as  branching. 

•  Narration,  dialog  and  sound  effects. 

•  Scoring  or  desired  feedback  from  user  interface,  selections,  or  actions. 


Typical  Storyboard  (Cold  Start) 


SW  S~«M1  *»  »M>WI  Pi 


J  NEW  ! 

Protecting  Weapon  Positions 

The  soft  launch  of  the  man-portable  Javelin  Close  Combat  Missile  provides  great  flexibility  in  urban  defense. 
However,  positions  must  be  <' 
for  the  missile,  Soldier  safety. 


deliberately  prepared  to  enhance  survrvability.  Additionally,  clear  overhead  flight  paths 
i,  and  backblast  hazard  must  be  considered  for  all  positions. 


Navigation 

Graphics. 

Animation 

Audio: 

v.dao:  Javeln  VO*o  wnw 
Function  a*  tv 
IntacactMty 


On  Screen  Treatments 

Open  wen  a  atfl  photo  of  the  gunner  facing  away  Play  me  v«Jeo 
(Javeln  Vdeo  wmvnsepctng  the  movement  of  the  gunner  to  the 
wrxlow  to  the  left  Pauae  and  naert  the  ati  photo  of  the  gunner 
teiwig  out  the  wndow  to  the  left  See  trigger  cues  n  Narrabon 


NJXT  go  to  IN_1_012 
BAC*  go  to  IN_1_010 


Identified  as  new  material 
vs.  reused  existing  material 


Narration:  (Open  with  still  photo  of  gunner  facing  away)  In  this  example,  a  Javelinteam 
has  a  primary  position  with  a  sector  of  fire  oriented  on  (say  letters)  T  R  P  two  to  the 
north,  (play  video)  but  the  gunner  can  quickly  shift  to  a  supplemental  position.  The 
supplemental  position  has  a  sector  of  fire  oriented  on  (say  letters)  T  R  P  one  to  the 
west  (stop  on  still  oriented  left).  Since  the  anti-tank  systems  are  priority  targets  for  the 
enemy,  both  positions  must  be  improved  to  enhance  survival.  Additionally,  backblast 
hazard,  safety  of  the  crew,  and  nearby  Soldiers  must  be  considered  for  both  positions. 

Figure  2.  A  typical  storyboard  used  for  new  material  development. 
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Figure  2  provides  an  example  of  a  typical  storyboard  page  used  for  developing  original 
materials  as  opposed  to  altering  existing  media.  These  pages  were  appropriately  labeled  to 
differentiate  new  materials  from  reused  materials  to  alert  the  programmers. 

Storyboard  formats  can  be  adapted  to  facilitate  the  reuse  of  existing  materials  into  new, 
revised,  point  of  need  lessons.  Our  reuse  of  existing  materials  fell  into  two  categories: 

•  Reuse  of  a  single  page,  and 

•  Reuse  of  multiple  pages  or  a  sections  (i.e.,  a  piece  of  training  covering  a  single  topic). 

Inclusion  of  existing  materials  frequently  required  minor  changes  such  as  image 
replacement,  modification  of  navigation  menus  or  action  controls  and  buttons,  modification  of 
on-screen  text,  or  modification  of  narrations.  Figures  3  and  4  provide  graphic,  annotated 
examples  of  the  storyboard  adaptations  we  used  to  develop  our  IMI  exemplars. 


Reuse  of  a  Single  Frame  with  Corrections/Adjustments 


Page  Audio:  In  this  lesson,  you  will  learn  how  to  prepare  positions  for  irtdiwdual-and 
crew-served  weapons  during  an  urban  operation.  The  key  training  points  for  this  lesson 
include  preparing  individual  rifle  positions,  preparing  machine  gun  positions,  and 
preparing  antitank  weapon  positions. 


From  the  Page  Audio:  Deleted  the  follow  words:  individual  and;  preparing  individual 
rifle  positions, 


1.  Frame  (page)  in  the  new 
lesson  is  indicated. 


2.  Changes  or 
adjustments  are 
indicated  in  the  text 
(in  red)  and  graphics 
of  the  storyboard. 


3.  Location  of  the  original 
material  is  indicated. 

4.  Adjustments  to  the 
narration  are  indicated  by 
line  through  (in  red)  and/or 
additions  to  a  transcript  of 
the  original  script. 


Figure  3.  A  storyboard  example:  reuse  of  a  single  frame  of  existing  IMI. 
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Reuse  of  Multiple  Frames  with  Minor  Corrections/Adjustments 


! CHUNK ! 


1.  Identified  as  a  “CHUNK”  (multiple 
frames)  of  existing  materials. 

2.  Frames  (pages)  in  the  new 
lesson  are  indicated. 


NOTE  Chan(C  imaje  on  pages  4  of  13. 
3  of  13.  and  7  of  13  to  mace  FA  1.1 


3.  Minor  changes  or 
adjustments  are 
indicated  in  the  text 
(in  red)  and  graphics 
of  the  storyboard. 


NOT!  Delete  or  cover  the  page  numbers 


Narration;  Nochangeto  bitting  For  4  of  13  through  12  of  13  From  page  12  of  13  forward  arrow/NEXT  returns  to  FA_1_002 


Navigation; 

Graph Soldier  with  raoo 

Audio; 

Video; 

Functionally 
interactivity 
tty  per  I  inis 


4.  Location  of  the  original 
material  is  indicated. 


5.  If  no  narration  adjustment  is  required  the  NOTES 
area  is  left  blank. 

Figure  4.  A  storyboard  example:  reuse  of  multiple  frames  of  existing  IMI. 

Care  was  exercised  to  minimize  extensive  changes  in  the  material  reuse.  Experience 
showed  that  although  storyboards  were  a  convenient  way  to  describe  modifications, 
programmers  at  times  had  difficulty  if  the  revisions  were  extensive.  For  extensive  changes,  it 
was  quicker  and  more  effective  to  assemble  the  modified  pieces  as  new  materials,  simplifying 
the  storyboard  for  the  programmer. 

Our  pedagogy  incorporated  features  described  in  research  by  Clark  and  Mayer  (2008), 
Merrill’s  (2002)  first  principles  of  instruction,  as  well  as  criteria  we  developed  in  the  first  phase 
of  this  research  (see  Blankenbeckler,  Graves,  &  Wampler,  2013;  also  Appendix  A).  We 
developed  storyboards  to  provide  complementary  visual  media,  on-screen  text,  narration,  and 
sound  effects.  Narrations  were  limited  to  18  seconds  or  less,  with  rare  exceptions  lasting  up  to 
30  seconds.  We  limited  narrations  to  help  learners  maintain  attention  on  the  IMI.  On-screen  text 
was  limited  to  no  more  than  one-third  (1/3)  of  the  display  area,  with  a  true  font  size  appearing  as 
12  point  or  larger  on  the  screen.  Images  and  diagrams  were  designed  to  be  clear  to  Soldiers  with 
normal  vision,  but  more  importantly  they  had  to  support  the  teaching  point.  All  images  were 
selected  for  a  specific  purpose.  Photos  were  selected  to  depict  a  Soldier  or  Soldiers  engaged  in 
actions  related  to  the  teaching  point,  or  a  scene  that  clearly  depicted  the  teaching  point.  Graphics 
used  standard,  common  military  map  symbols  or  graphics.  Obscure  or  less  common  selections 
were  given  a  legend  or  explanation.  Diagrams  and  supporting  graphics  were  fabricated  to 
support  the  teaching  point.  When  needed  (i.e.,  photographs  could  not  be  located  or  a  specific 
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Soldier  posture  or  example  was  required),  VBS2  was  used  to  build  the  scene.  VBS2  also  was 
used  to  create  video  segments  as  examples  or  models  of  an  action  or  teaching  point. 

To  address  our  concern  as  to  how  to  present  the  adjust  fire  task  as  it  occurs  in  time,  we 
used  VBS2  to  capture  successive  graphics  of  the  effects  of  rounds  exploding  at  predefined 
locations,  as  well  as  the  depiction  of  targets  in  realistic  situations.  While  each  individual 
storyboard  received  significant  attention,  they  were  designed  to  contribute  to  a  section  of 
content,  often  referred  to  as  a  “chunk.”  Chunking  is  a  technique  that  groups  items  of  content  in 
terms  of  their  relatedness  within  a  task  or  knowledge  domain  (Sweller,  1988).  A  content 
chunking  approach  provides  flexibility,  complexity,  and  depth  to  the  information  presented. 
Chunking  in  this  way  contributed  to  the  logical  flow  of  our  material,  presenting  it  in  a  cohesive 
manner.  The  graphics  and  video,  narrations,  and  text  were  integrated  into  training  pages  or 
frames.  These  pieces  were  programmed  in  Hyper  Text  Markup  Language  (HTML)  creating  web 
pages  that  could  be  accessed  through  standard  web  browser. 

A  further  concern  arose  when  developing  the  diagnostic  assessments  used  specifically  for 
the  tailored  training  IMI  and  the  think-ahead,  check-on-learning  assessments  used  in  each  of  the 
point  of  need  IMI  variations.  The  IMI  needed  to  be  capable  of  providing  learners  with 
meaningful  feedback  in  order  for  it  to  be  effective.  The  pre-  and  post-assessments  used  for  the 
tailored  training  IMI  exemplars  required  greater  specificity  than  the  think-ahead  and  check-on- 
learning  assessments.  The  pre-  and  post-assessments  were  used  to  guide  learners  to  select 
training  paths  based  on  their  demonstrated  proficiencies.  Figure  5  presents  an  example  of  the 
programming  instructions  we  used  to  structure  the  feedback  for  multiple  response  questions. 

Tailored  Training:  Conduct  a  Defense  by  a  Squad  in  an  urban 

Operation 

IN_3 

Assessment  Feedback ;  in _3  Q20.1  .  _ 

21  Detailed  Programming  for  the  Assessment  Feedback  Chart 


Field  A: 

On  frame  IN_3_012,  if  C  is  selected,  store  one  "Go"  for  field  A  of  IN_3_020. 

On  frame  IN_3_016,  if  D  is  selected,  store  one  "Go"  for  field  A  of  IN_3_020. 

If  two  "Go's"  are  stored  for  A,  a  0  will  be  placed  in  Field  A  of  IN_3_020.  If  less  than  two  "Go's"  are  stored,  a  Bwill  be  placed  in  Field  A. 


Field  B: 

On  frame  IN_3_009,  if  B,C,  D,  E,  &  F  are  aN  selected,  store  one  "Go"  for  field  Bof  IN_3_020. 

On  frame  IN_3_011,if  Bis  selected,  store  one  "Go"  for  field  Bof  IN_3_020. 

On  frame  IN_3_014,  if  A,  B,  C,  D,&  E  are  aN  selected,  store  one  "Go"  for  field  Bof  IN_3_020. 

On  frame  IN_3_014,  if  A,  B,  C,  &  D  are  aN  selected,  store  one  "Go"  for  field  Bof  IN_3_020. 

If  four  "Go's"  are  stored  for  B,  a  0  will  be  placed  in  Field  B  of  IN_3_020.  If  less  than  four  "Go's"  are  stored,  a  Bwill  be  placed  in  Field  B. 
Field  C: 

On  frame  IN_3_017,if  Cis  selected, storeone  "Go"  forfield  CoflN_3_020. 

On  frame  IN_3_016,  if  D  is  selected,  store  one  "Go"  for  field  Cof  IN_3_020. 

If  two  "Go's”  are  stored  for  C,  a  0  will  be  placed  Field  C  of  IN_3_020.  If  less  than  two  "Go's"  are  stored,  a  Bwill  be  placed  in  Field  C. 


Field  D: 

On  frame  IN_3_010,  if  B,C,  &  E  are  selected,  store  one  "Go"  for  field  Dof  IN_3_020. 

On  frame  IN_3_013,  if  A  is  selected,  store  one  "Go"  for  field  Dof  IN_3_020. 

On  frame  IN_3_019,  if  C  &  Dare  both  selected,  store  one  "Go" for  field  Dof  IN_3_020. 

If  three  "Go's"  are  stored  for  D,  a  0  will  be  placed  Field  Dof  IN_3_020.  If  less  than  three  "Go's"  are  stored,  a  Bwill  be  placed  in  Field  D. 


Figure  5.  Example  instructions  to  the  programmer  for  providing  feedback  to  learners. 
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Figure  6  presents  an  example  of  the  programming  instructions  we  used  to  guide  learners 
in  training  after  the  pre-assessment. 

Tailored  Training:  Conduct  a  Defense  by  a  Squad  in  an  Urban 

Operation 

IN_3 

Assessment Feedback;  in 13  CBO.2  .  _ 

21  Detailed  Programming  for  the  Assessment  Feedback  Chart 

If  all  fields  are  0  (green  checks)  enable  NEXT.  Selection  of  NEXT  will  go  to  the  conclusion  of  training  at  IN_3_109.  Do  not 
Enable  NEXT  if  a  0  NO  GO  is  received. 

If  any  0  NO  GO  boxes  appear  in  fields  A,  B,  C,  or  D,  illuminate  (green  fill)  the  "Recommended  Training  Topic"  indicator 
beside  the  button  indicated  based  on  the  question/frame  receiving  the  NO  GO: 


Training 

Button 

IMI  Frame  Number 

IN _ 3 _ ? 

009 

010 

Oil 

012 

013 

014 

015 

016 

017 

018 

019 

E 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

F 

X 

X 

X 

X 

X 

X 

G 

X 

X 

X 

X 

X 

X 

H 

X 

X 

J 

X 

X 

The  student  may  select  E,  F,  G,  H,  or  J  forretraining  or  additional  training.  All  retraining  will  return  to  IN_3_020. 

Ghost  and  do  not  permit  selection  of  button  K  Post-Training  Assessment  until  after  the  student  selects  additional  training, 
buttons  E,  F,  G,  FI,  or  J .  Once  enabled.  Navigation  is  as  indicated:  If  button  E  is  selected,  go  to  IN_3_021.  If  button  F 
is  selected,  go  to  IN_3_  054.  If  button  G  is  selected,  go  to  IN_3_073.  If  button  H  is  selected,  go  to  IN_3_082. 

If  button  J  is  selected,  go  to  IN_3_085.  When  button  K  is  selected,  begin  the  Post-Training  Assessment  at 
IN_3_093. _ 

Figure  6.  Example  instructions  to  the  programmer  to  provide  training  guidance  to  the  learner. 

We  used  a  multi-phased  system  to  review  and  approve  the  storyboards.  First,  the 
developers  constructed  the  storyboards,  followed  by  an  internal  review.  This  initial  review  was 
to  ensure  accurate  content  and  up-to-date  doctrine,  etc.  Storyboards  were  then  produced  through 
an  iterative  process  of  development,  review,  and  comment  between  military  subject  matter 
experts  and  the  Government.  Once  all  members  of  the  research  team  were  satisfied  with  the 
content  and  design,  the  edited  storyboards  were  provided  to  graphic  artists.  These  artists  refined 
the  graphics  and  text.  Finally,  the  materials  were  provided  to  the  programmers.  After  the 
programmers  assembled  the  materials,  a  production  review  was  used  to  assure  desired  navigation 
and  functionality.  The  Alpha  version  of  each  IMI  module  was  returned  to  the  developer  for 
review  to  assure  correct  translation  of  intent,  compliance  of  the  graphics  with  desired 
appearance,  and  navigation  of  or  reaction  to  the  software.  Researchers  then  reviewed  the  Alpha 
version  with  the  list  of  corrections  and  edits  identified  by  the  development  team.  These 
corrections  and  edits  were  provided  to  the  artists  and  programmers.  The  corrected  copy  became 
the  Beta  version,  which  was  returned  to  the  developer  for  a  similar  review  and  comment  by  our 
research  team.  After  the  final  reviews  and  corrections,  the  development  was  complete.  This 
cyclic,  iterative  review  was  intended  to  assure  a  quality  product  for  the  learners. 
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The  Structure  of  the  IMI  Modules 


Adjust  Indirect  Fire.  For  two  of  the  three  Adjust  Indirect  Fire  modules,  the 
instructional  infonnation  and  activities  were  arranged  in  sequence,  following  the  order  of  steps  in 
the  Adjust  Indirect  Fire  task  (see  U.S.  Anny,  2008).  Each  step  in  this  procedure  depends  on  the 
previous  steps.  For  example,  doctrine  stipulates  that  the  standard  indirect  fire  mission  takes 
place  in  three  transmissions.  Once  the  initial  adjusting  round  impacts,  the  data  for  adjustments 
are  transmitted.  These  adjustments  close  in  on  the  target’s  location  before  the  final  call  to  ‘fire- 
for-effect.’  Doctrine  also  stipulates  the  transmission  to  end  the  mission.  Standardization  helps 
users  to  communicate  effectively  with  an  indirect  fire  unit’s  fire  direction  center  (FDC)  and  to 
obtain  the  desired  effects  against  a  target. 

The  core  IMI  module  for  Adjust  Indirect  Fire  differed  from  the  familiarization  and 
tailored  training  IMI  modules  only  in  that  it  focused  on  the  immediate  suppression  mission,  a 
common  mission  that  is  initiated  with  an  abbreviated  call  for  fire.  The  immediate  suppression 
mission  is  an  emergency  fire  mission  designed  to  quickly  obtain  suppressive  fires  on  a 
threatening  target.  Only  essential  information  is  needed  for  a  single  transmission  to  the  FDC. 
Training  materials  were  focused  only  on  how  to  transmit  an  immediate  suppression  mission;  they 
were  not  compared  with  the  standard  call  for  fire.  The  standard  procedure  for  transmitting 
adjustment  data  was  included,  as  was  the  specific  information  to  conclude  the  mission.  The 
reasons  for  providing  a  target  description  were  reviewed,  although  it  is  not  a  required  element  of 
the  mission.  Materials  were  structured  in  chunks,  following  the  sequence  of  immediate 
suppression  mission. 

Conduct  a  Defense  by  a  Squad.  The  three  modules  for  the  Conduct  a  Defense  by  a 
Squad  were  a  greater  challenge  than  the  Adjust  Indirect  Fire  modules.  When  conducting  a  squad 
defense,  a  leader  is  engaged  in  complex  cognitive  tasks  when  planning  and  executing  an  urban 
defense,  positioning  subordinate  elements  and  supporting  weapons  (U.S.  Army,  2008).  The 
leader  has  to  track  multiple  variables,  make  sound  judgments  about  many  issues,  and  supervise 
subordinates’  concurrent  individual  and  collective  tasks.  Further,  a  leader  must  adhere  to 
standard  procedures,  although  the  order  of  steps  and  measures  of  procedures  are  not  fixed.  In 
this  task,  some  steps  may  be  omitted  and  others  modified  to  save  time.  Altering  some  steps  often 
has  no  detrimental  effect  on  this  mission.  Given  the  time  pressures  in  conducting  a  defense, 
work  has  to  be  prioritized  to  enable  it  to  be  successfully  executed.  For  example,  positions  and 
fields  of  fire  for  machine  guns  and  anti-tank  weapons  must  be  examined,  proofed,  and  adjusted 
quickly  to  prevent  wasted  time,  effort,  and  to  allow  time  for  improvements. 

All  three  of  these  IMI  modules  address  primary,  alternate,  and  supplementary  positions. 
The  familiarization  IMI  module  employed  training  materials  structured  in  chunks  to  address  each 
of  these  positions.  Some  sections  address  how  to  place  units  or  weapons  in  optimum  locations 
and  control  fires.  Other  sections  address  how  to  prepare  positions.  The  tailored  training  IMI 
module  used  assessments  that  asked  the  learner  to  evaluate  the  decisions  and  weapon  positions  of 
a  hypothetical  peer.  The  blocks  of  training  were  designed  to  address  how  to  prepare  positions, 
place  units  or  weapons  in  optimum  locations,  and  control  fires. 
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The  core  IMI  module  introduced  general  characteristics  of  the  Javelin  Close  Combat 
Missile  (CCM)  system.  Initial  instruction  focused  on  the  Javelin’s  capabilities  and  limitations 
for  firepower,  maneuver,  and  protection.  Subsequent  instruction  focused  on  how  to  employ  the 
Javelin  CCM.  Employment  considerations  are  presented  in  the  context  of  a  tactical  scenario 
providing  contrasts  of  more,  or  less,  viable  tactical  judgments.  Principles  of  Fire  Control  serve 
as  the  context  concerning  position  selection  and  controlling  fires.  Training  content  was 
organized  within  the  module  to  guide  new  learners  to  correct  weapon  preparations  and 
placement. 

Developing  Diagnostic  Assessments  for  Tailored  Training  IMI.  A  key  feature  of  the 
two  tailored  training  IMI  modules  was  feedback  from  the  pre-  and  post-training  assessments.  In 
the  Adjust  Indirect  Fire  module,  the  learner  was  provided  performance  feedback  on  each  task 
step.  Figure  7  shows  a  sample  feedback  display  for  the  Adjust  Indirect  Fire  task. 

Assessment  Feedback 


Call  for  Fire 

Adjustment 

Formulate  a  Call 
for  Fire 

Determine/Transmit 
Elements  of  a  Call 
for  Fire 

Adjust  Rounds 
on  Target; 

End  the  Mission 

Determine  type  of 
mission  and  method 
of  target  location. 

c 

Requester 
identification  and 
warning  order 

c 

Determine  / 

transmit 

corrections 

C 

C 

c 

Prepare  for 
adjustments. 

w 

Target  location 

c 

Enter  Fire 
for  Effect 

w 

Determine  elements 
of  the  Call  for  Fire. 

c 

Target  description 

c 

Determine 

Effects 

W 

OT  direction 

w 

End  the  Mission 

C 

If  all  categories  are  a  GO,  congratulations,  you 
have  attained  the  standards  for  this  lesson  and  may 
end  training.  If  only  one  topic  in  the  Call  for  Fire  or 
Adjustment  portion  of  the  Assessment  is  a  NO  GO, 
it  is  recommended  that  you  select  Fundamental 
training  for  the  topic  or  topics  indicated. 

If  more  than  two  topics  receive  a  NO  GO,  it  is 
recommended  that  you  select  the 

Comprehensive  Call  for  and  Adjust  Fire 
Training. 


Assessment  Scoring: 


=  NO  GO 


=  GO 


Figure  7.  Feedback  format  for  both  the  Pre-  and  Post-Training  Assessments  for  the  Adjust 
Indirect  Fire  tailored  training. 

The  pre-  and  post-assessments  recommended  to  Soldiers  ways  in  which  they  could 
address  identified  shortcomings  in  their  understanding.  Learners  were  given  training 
recommendations  related  to  the  primary  navigation  menu  for  the  module.  This  menu  is  shown  in 
Figure  8. 
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Training  Navigation 


Fundamental  Call  for  Fire  Training 


Fundamental  Adjustment  Training 


Comprehensive  Call  for  Fire  and 
Adjust  Fire  Training 


If  only  one  topic  in  the  Call  for  Fire  or  Adjustment  portion  of  the  Assessment  is  a  NO  GO,  it  is  recommended  that 
you  select  Fundamental  training  for  the  topic  or  topics  indicated.  If  more  than  two  topics  receive  a  NO  GO,  it  is 
recommended  that  you  select  the  Comprehensive  Call  for  and  Adjust  Fire  Training. 


Figure  8.  Primary  Training  Navigation  Menu. 

Note:  the  guidance  from  the  Assessment  Feedback  page  is  repeated. 


For  the  Conduct  a  Defense  by  a  Squad,  the  assessments  were  designed  differently.  The 
feedback  was  focused  on  the  three  principle  subject  areas  addressed  by  the  modified  task.  Based 
on  how  learners  performed  on  the  assessment,  recommendations  were  made  concerning  the 
various  training  options.  These  recommendations  were  shown  in  the  lower  right  comer  of  the 
assessment  feedback  screen  (see  Figure  9). 


Like  the  tailored  Adjust  Indirect  Fire  module,  the  elements  of  the  feedback  related 
directly  to  the  primary  training  navigation  menu  for  the  module.  While  feedback  was  only 
provided  for  three  principle  subject  areas,  the  training  recommendations  were  based  on  the 
assessment  questions  that  were  incorrect.  Some  questions  overlapped  subject  areas  and  resulted 
in  multiple  training  recommendations. 


Assessment  questions  for  the  Adjust  Indirect  Fire  module  were  answered  and  scored  as 
single  responses.  However,  the  Conduct  a  Defense  by  a  Squad  assessment  questions  contained 
both  single  and  multiple  response  questions.  A  sample  of  one  of  the  multiple  response  questions 
is  provided  in  Figure  10. 


20 


If  all  categories  are  a  GO,  congratulations,  you 
have  attained  the  standards  for  this  lesson  and 
may  end  training  or  elect  to  take  selected  topics. 
If  you  elect  to  end  training,  select  NEXT  to  EXIT. 

If  any  elements  of  the  task  was  assessed  as  a 
NO  GO,  the  Recommended  Training  Topic  or 
Topics  have  been  illuminated.  Upon  completion 
of  additional  training  you  will  be  routed  to  the 
Post-Training  Assessment. 

Suggested  sections  you  should  review: 

•  Platoon  and  Squad  Defensive  Planning 

•  Selecting  Weapon  Positions  and 

Controlling  Fires 


Figure  9.  Feedback  format  for  both  the  Pre-  and  Post-Training  Assessments  for  the  Conduct  a 
Defense  by  a  Squad  tailored  training. 


1.  Designate/prepare/adjust 

2.  Designate/prepare: 

primary  individual/team/ 

-  Alternate  positions 

a 

crew  fighting  positions  to: 

-  Supplementary 

positions 

id 

(These  positions  should 

Provide  effective/ 

meet  the  same  standards 

adequate  fields  of  fire. 

as  primary  positions.) 

3.  Employ  fire  control 

a 
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measures  effectively 

Enhance  Soldier/unit 
survival  and  safety. 

a 
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a 

^i 

Assessment  Scoring: 

a- 

GO  ^]  =  NO  GO 

SAW  Position  Selection  and  Planning 


The  M240B  MG  Crew  in  your  sector  is  assigned  an  FPL  critical  to  the  Platoon’s 
Fire  Plan.  You  have  positioned  your  in  your  Alpha  Fire  Team’s  M249  SAW  to  be 
able  to  reinforce  the  fires  of  this  MG  crew  and  cover  the  FPL  during  barrel  changes 
and  unforeseen  stoppages.  What  characteristics  of  the  M249  SAW  must  be 
considered  when  assigning  this  mission?  Select  all  that  apply,  then  click  SUBMIT 
to  record  the  answers. 


ie  normal  basic  load  is  1 ,000  rounds,  additional  ammunition  will  be 
required  to  sustain  fire. 


□ 
□ 


Grazing  fire  is  only  effective  out  to  400  meters  using  the  tripod  with  the 
T&E  on  level  or  sloping  terrain. 


The  weapon's  fixed  barrel  eliminates  changes  but  increases  cooling  time 
during  sustained  firing. 

he  Sustained  and  Rapid  Rates  of  Fire  for  the  M249  are  about  half  of 
those  expected  for  the  M240B. 


ie  M249  is  an  individual  weapon;  the  M240B  is  crew-served.  For 
sustained  fires,  another  squad  member  may  be  required  to  assist. 


Figure  10.  A  sample  multiple  response  question  from  the  Post  Training  Assessment  for  the 
Conduct  a  Defense  by  a  Squad  tailored  training. 


The  diagnostic  assessments  for  Adjust  Indirect  Fire  used  a  tactical  scenario,  which 
required  fire  engagement  and  adjustments  on  an  enemy  target.  We  used  screen  shots  from  the 
VBS2  software  to  reflect  imagery  required  during  specific  sequences.  VBS2  enabled  us  to 
present  realistic  target  images,  the  effects  of  the  artillery  rounds,  and  the  placement  of  targets  and 
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effects  to  replicate  what  is  required  for  the  Adjust  Indirect  Fire  task.  The  same  sequence  of  task 
steps  and  real-world  problem  solving  was  used  within  the  training  modules.  VBS2  was  useful  in 
creating  realistic  still  images  of  the  battlefield,  including  high  quality  impact  effects. 

Our  diagnostic  assessments  for  the  Conduct  a  Defense  by  a  Squad  also  used  a  scenario 
based  format.  Two  basic  scenarios  were  used  where  the  Soldier  played  the  role  of:  (a)  a  squad 
leader  replacing  the  leader  who  had  established  the  defensive  positions,  or  (b)  a  leader  correcting 
and  refining  the  positions  that  a  subordinate  had  established.  In  both  situations,  the  learner  was 
provided  diagrams  of  the  broader  defense  and  unit  positions  to  provide  context  for  the  scenarios. 
Within  training  modules,  we  used  shortened  scenarios  to  cover  a  larger  number  of  concepts  and 
principles.  All  assessments  and  sections  of  training  were  based  on  real-world  tactical  problems 
and  applications  of  principles  in  combat.  The  training  encouraged  learners  to  exercise  tactical 
judgment,  supporting  their  development  into  their  future  role  as  squad  or  fire  team  leaders. 

Incorporating  Additional  Resources.  To  make  the  IMI  useful  to  learners  who  would 
like  to  pursue  their  learning  in  greater  depth,  we  provided  access  to  additional  training  materials. 
In  each  of  the  modules,  additional  resources  were  made  available  through  a  “References”  tab. 
The  references  included  all  available  doctrinal  publications  related  to  the  tasks  being  trained.  In 
addition,  the  relevant  sections  of  the  documents  were  provided  for  quick  reference. 

An  example  of  the  references  for  the  Adjust  Indirect  Fire  training  is  provided  in  Figure 
1 1 .  The  references  for  Conduct  a  Defense  by  a  Squad  lessons  were  more  extensive,  given  the 
variety  of  tasks  associated  with  conducting  a  defense.  The  references  are  available  from  all 
pages  in  all  modules  for  learners  who  desire  additional  details. 
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Resources 


O 


Resources  and  References  for  Task  061-283-6003:  Adjust  Indirect  Fire 

The  references  below  may  assist  your  learning  and  will  provide  additional  details 
on  aspects  of  the  task  and  the  related  skills.  Make  note  of  the  chapter  or  pages 
that  you  wish  to  read  or  review.  Click  on  the  reference  to  open  the  document  to  the 
cover  page. 

STP  21-24-SMCT,  Soldier's  Manual  of  Common  Tasks.  Warrior  Leader  Skills  Level 

2.  3.  and  4.  September  2008 

•  Chapter  3  Warrior  Leader  Skills  Level  2,  3,  and  4  Tasks 
061-283-6003,  Adjust  Indirect  Fire,  page  3-46 

FM  3-09.30,  Tactics.  Techniques,  and  Procedures  for  Observed  Fire  and  Fire 

Support  at  Battalion  Task  Force  and  Below  (Final  Draft).  31  May  2001 

•  Chapter  6  Call  for  Fire 

•  Chapter  7  Adjustment  of  Fire 

FM  3-21.8,  The  Infantry  Rifle  Platoon  and  Sauad.  March  2007 

•  Chapter  2  Employing  Fires 

o  Section  III  -  Engaging  the  Enemy  without  Engaging 
FriendlyTroops,  page  2-9 
o  Section  IV  -  Employing  Indirect  Fires,  page  2-27 

•  Appendix  C  Fire  Planning 

TC  3-21.8,  Infantry  Rifle  and  Mechanized  Platoon  Collective  Task  Publication. 

(undated) 

•  Chapter  2  Crosswalk  and  Outlines 

o  Conduct  an  Attack  by  Fire  (Platoon  -  Company), 

07-2-1 256,  page  2-12 

o  Integrate  Indirect  Fire  Support  (Platoon  -  Company), 
07-2-3036,  page  2-151 


Figure  11.  Adjust  Indirect  Fire  References  provided  in  the  IMI  modules. 
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Pulling  the  Pieces  Together 


One  of  the  major  hurdles  we  encountered  in  this  research  concerned  how  to  reuse  existing 
Anny  IMI.  Given  the  variety  of  IMI  available  and  the  different  ways  in  which  it  can  be 
developed,  deployed,  and  archived,  we  had  to  use  a  variety  of  software  tools  and  programming 
languages.  A  variety  of  tools  were  used  to  extract,  develop,  correct,  and  program  materials  for 
this  effort  (see  Table  4). 


Table  4 

Software  Tools  and  Programming  Languages  Used  to  Build  the  IMI  Modules 


Tool 

Examples 

Software 

Adobe  Flash  (ActionScript  2  &  3) 

Adobe  Dreamweaver  (HTML  &  JavaScript  &  CSS) 

Adobe  Photoshop  (graphics) 

Adobe  Captivate 

Camtasia  Studio  (screen  capture  software) 

Snag-it 

Programming  Languages 

ActionScript  2  &  3 

HTML 

JavaScript 

CSS 

Note:  HTML  refers  to  Hypertext  Markup  Language;  CSS  refers  to  Cascading  Style  Sheets. 


We  had  to  develop  some  workaround  solutions  to  problems  we  encountered  when  trying 
to  reuse  parts  of  an  existing  IMI.  As  indicated  in  our  initial  report  (see  Blankenbeckler,  Graves, 
&  Wampler,  2013),  there  was  no  centralized  database  or  learning  content  management  system 
(LCMS)  from  which  we  could  draw  IMI  materials  for  reuse.  For  almost  every  case,  we  had  to 
trace  IMI  to  the  proponent  that  had  developed  and  retained  the  original  source  files. 

In  some  cases,  the  source  files  were  incomplete  and  not  viable  for  reuse.  A  few  of  the 
existing  lessons  had  all  source  materials,  including  the  storyboards  and  complete  instructional 
media  design  packages  (IMDPs)  (TRADOC,  2003).  Those  with  complete  IMDPs  were  at  times 
inaccurate  in  their  verbal  descriptions  of  design  and  instructional  strategies.  While  some  did 
provide  complete  descriptions  of  the  training  program,  lesson  development  support  information, 
and  wire-frame  diagrams  that  identified  the  Terminal  Learning  Objectives  (TLOs),  Enabling 
Learning  Objectives  (ELOs),  and  SCOs  that  could  support  reuse  decisions,  a  detailed  and  careful 
review  of  the  materials  in  the  programmed  IMI  was  required. 

Having  a  consistent  pedagogy  was  a  critical  factor  in  our  reuse  decisions.  The  graphic, 
narration,  or  segment  we  were  considering  had  to  fit  our  purpose  and  approach.  In  a  few  cases, 
we  were  able  to  modify  the  instructional  approach  to  accommodate  the  materials  to  be  reused. 
Review  of  the  existing  IMI  was  also  required  to  assure  that  equipment,  unifonns,  doctrinal  tenns, 
as  well  as  tactics,  techniques,  and  procedures  (TTP)  portrayed  were  current  and  not  distracting  to 
the  learner. 
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Once  training  tasks  were  selected,  a  more  detailed  review  of  the  related  IMI  was 
undertaken  to  identify  the  precise  elements  desired  for  reuse.  This  process  resulted  in  mapping 
the  existing  IMI  to  identify  the  desired  elements.  This  mapping,  while  time  consuming,  proved 
most  valuable  to  the  graphic  artist  and  programmers  who  extracted  or  captured  the  desired 
materials.  The  target  element  could  be  an  entire  module  or  sub-element  of  an  existing  lesson,  a 
short  animation,  a  demonstration,  a  single  image,  a  check-on-leaming,  or  an  assessment. 

The  FCoE’s  materials — specifically  for  Adjust  Indirect  Fire — were  the  easiest  to  work 
with  to  extract  sections  and  to  adapt  to  the  pedagogy  we  used  for  all  three  of  the  point  of  need 
lessons.  The  source  files  contained  much  of  the  desired  material.  To  a  great  extent,  existing 
animations,  on  screen  text,  transition  effects,  and  narrations  were  reusable  or  were  easy  to  edit. 
Some  minor  alterations  were  required: 

•  We  had  to  replace  some  graphics  because  they  mixed  woodland  battle  dress  uniform 
(BDU)  and  desert  BDU,  which  preceded  the  current  Army  Combat  Uniform  (ACU). 
Mixed  uniforms  were  often  observed  in  the  early  years  of  our  conflicts  in  Iraq  and 
Afghanistan.  These  were  replaced  with  Soldiers  in  ACU  (see  Figure  12). 
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Figure  12.  An  example  of  image  substitution  in  reused  IMI.  The  new  image  provided  an  action 
photo  of  a  battle-ready  Soldier  in  the  current  ACU  to  replace  the  existing,  more  passive  image  of 
a  Soldier  in  the  outdated  BDU. 

•  Some  minor  issues  needed  to  be  corrected  concerning  target  location  and  grid  coordinate 
infonnation  displayed  in  on-screen  text  boxes  or  vocalized  in  the  narration.  The  on¬ 
screen  text  boxes  were  replaced  or  edited.  New  narrations  were  dubbed-in  to  reflect 
current  doctrine  (see  Figure  13). 
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New  Narration:  A  cal!  for  fire  is  a  concise  message  prepared  by  an  observer.  It  is  a  re  quest  for  fires  notan  order. 
Graphic  Correction:  Change  thetext  inthe  pop  up  box.  "Grid  642515 "should  read  “QA  642515". 


Figure  13.  An  example  of  edits  to  on-screen  text  in  reused  IMI. 

Note:  The  correction  reflected  current  doctrine,  omitting  the  word  “grid”  and  providing  the 
100,000  meter  grid  square  identification  as  part  of  the  target  location  to  be  transmitted. 

•  We  framed  some  animations  to  place  on-screen  navigation  buttons.  These  adjustments 
assured  that  existing  graphics  and  text  pertinent  to  training  were  displayed  properly. 

•  We  edited  some  narrations  to  account  for  the  revised  timing  of  animations. 

That  said,  much  of  the  FCoE’s  existing  material  was  reusable  with  only  minor 
adjustments  for  the  familiarization  IMI  module.  To  a  lesser  extent,  this  remained  true  in  the  core 
IMI  module  that  was  developed.  The  pre-  and  post-tests  and  comprehensive  Adjust  Indirect  Fire 
training  developed  for  the  tailored  training  IMI  module  were  designed  without  reuse  of  existing 
materials.  However,  optional  training  blocks  in  the  tailored  training  IMI  module  used  content 
from  the  familiarization  and  core  IMI  training  modules. 

IMI  concerning  infantry  leader  skills  and  common  tasks  were  provided  by  the  Iowa 
National  Guard  Distributed  Learning  Development  Center.  These  were  the  oldest  of  the  source 
materials  we  collected.  Due  to  the  age  of  the  materials,  some  were  unusable  because  the  source 
code  had  been  superseded  by  newer  versions  of  software  and  were  no  longer  compatible.  In 
these  cases,  we  were  able  to  capture  video  and  audio  recordings  of  the  IMI  running  on  Army 
Knowledge  Online  (AKO)  My  Training  Tab  (MT2)  site.  Using  multi-track  recording,  we  were 
able  to  capture  video  and  audio  on  separate  tracks  to  enable  us  to  later  edit  each  track 
independently. 
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The  only  notable  difference  in  the  way  the  captured  media  perfonned  was  that  the  video 
segments  had  to  play  completely  before  navigation  buttons  became  active.  Learners  who  desired 
to  speed  through  the  materials  encountered  some  delays  while  waiting  for  segments  to  finish 
playing.  In  addition,  getting  the  video  and  audio  to  synchronize  was  initially  an  issue,  but  this 
was  easily  resolved  with  audio  editing  programs.  Some  existing  material  was  reusable  with  only 
minor  adjustments,  such  as  changing  the  placement  of  machine  guns  and  anti-tank  systems  used 
in  one  of  the  familiarization  IMI  modules. 

In  summary,  the  team  was  able  to  reuse  and  integrate  images,  animation  and  video 
segments,  menus,  and  narrations  in  five  of  the  modules.  However,  in  the  core  Conduct  a 
Defense  by  a  Squad  IMI  module  (Designate  and  Prepare  Urban  Fighting  Positions  for  a  Javelin 
Team),  editing  proved  far  too  complex  and  time  consuming.  The  resulting  animations  and 
narrations  were  choppy  and  did  not  seem  to  match  well  with  the  selected  methods  of  instruction. 
A  very  limited  number  of  images  were  extracted  and  reused  in  this  module.  Recall  that  about 
30%  of  the  final  point  of  need  IMI  modules  were  created  from  existing  IMI,  while  70%  had  to  be 
developed  as  new  material. 

We  did  not  use  a  LMS  to  run  the  IMI  exemplars,  although  they  could  be  incorporated 
into  such  a  system.  Instead,  Hypertext  Markup  Language  (HTML)  was  used,  enabling  the  IMI 
exemplars  to  run  in  a  web  browser.  While  this  approach  precluded  some  data  collection 
techniques  and  required  additional  programming  for  the  pre-  and  post-assessments  in  the  tailored 
training  IMI  exemplars,  its  “low  tech”  design  maximized  compatibility  across  systems.  Using 
HTML  also  minimized  the  unintended  effects  of  code  that  remained  from  the  previous  IMI  or 
LMS.  These  left-behind  elements  tended  to  negatively  affect  timing  or  presentation  of  content. 

Left-behind  elements  often  introduced  problems  with  navigation.  These  problems  were 
overcome  by  establishing  additional  navigation  selections  within  the  instructions  frame.  The 
supplemental  navigation  buttons  were  made  a  standard  across  all  six  exemplars.  While  generally 
intuitive,  some  learners  initially  were  frustrated  by  this  multi-level  navigation  scheme. 
Supplemental  buttons  were  only  present  when  needed  and  the  next  (forward)  arrows  flashed 
when  the  narration,  animation,  or  on-screen  actions  were  completed  to  provide  a  visual  cue. 
During  the  assessments,  some  learners  did  experience  navigation  issues,  occasionally  ending  up 
where  they  had  not  intended  to  be.  With  minor  assistance,  all  learners  were  able  to  use  the 
navigation  effectively. 

Instructional  Design  and  Point  of  Need 

Early  in  this  research,  we  focused  on  instructional  design  issues  over  those  of 
technology — although  both  issues  are  important  in  the  point  of  need  context.  One  major  factor 
in  instructional  design  is  detennining  an  audience’s  learning  needs  and  then  developing 
instruction  to  meet  those  needs  (Clark  &  Mayer,  2008).  Another  focuses  on  issues  surrounding 
how  to  maintain  quality  of  instruction  when  implementing  it  with  particular  technologies. 

Problems  can  arise  when  technology  is  emphasized  over  the  selection  and  presentation  of 
training  content  (Clark  &  Mayer,  2008).  A  piece  of  training  can  be  well-designed  for  one 
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technology,  but  poorly-designed  for  another.  Not  all  features  of  instructional  design  are 
transferable  across  technologies.  For  example,  the  fonts,  font  sizes,  graphics,  audio,  and  video 
media  used  in  a  design  are  in  part  determined  by  the  specifications  of  the  intended  technology, 
i.e.,  screen  size,  processing  power,  or  availability  of  bandwidth  to  support  applications.  A  piece 
of  IMI  designed  to  be  viewed  on  a  computer  screen  is  not  optimally  viewed  on  a  smartphone. 

Rather  than  focus  solely  on  how  to  apply  new  technology  in  educational  and  training 
settings,  Mayer’s  research  (see  Mayer,  2003;  Clark  &  Mayer,  2008;  Mayer,  2009)  has  sought  to 
identify  general  principles  for  good  multimedia  instructional  design.  Accordingly,  concerns  with 
instructional  design  must  always  take  precedence  over  how  to  incorporate  new  educational 
technologies.  For  instance,  the  design  features  of  IMI  that  are  optimal  for  learning  are  rarely 
those  that  require  a  lot  of  bandwidth.  In  fact,  when  a  training  developer  overemphasizes 
technology,  the  actual  content  and  desired  learning  outcomes  may  be  lost  to  all  the  “bells  and 
whistles.” 

The  critical  features  of  effective  instructional  design  reduce  irrelevant  information  and 
help  learners  think  more  deeply  about  the  training  domain.  According  to  Clark  and  Mayer 
(2008),  these  critical  design  features  address  three  goals:  (a)  reducing  extrinsic  cognitive 
processing,  (b)  managing  intrinsic  cognitive  processing,  and  (c)  facilitating  generative 
processing.  Table  5  summarizes  the  features  of  well-designed  IMI  identified  by  Mayer  and 
colleagues  (see  Veronikas  &  Shaughnessy,  2005;  Mayer,  2009;  Moreno  &  Mayer,  2007;  Clark  & 
Mayer,  2008). 


Table  5 

Features  of  Well-Designed  IMI  from  Mayer  and  Colleagues 


Goal 

Features 

Description 

To  Reduce 
Extrinsic 
Cognitive 
Processing 

Coherence 

Signaling 

Eliminating  extraneous  words,  pictures,  images 

Highlighting  important  words  (e.g.,  section  headings,  highlighting, 
boldface  font) 

Redundancy 

Combining  animations  with  narrations  rather  than  animation, 
narration  and  text 

Spatial  Contiguity 

Placing  corresponding  portions  of  pictures  and  words  near  each 
other 

Temporal  Contiguity 

Presenting  corresponding  animation  and  narration  simultaneously 
rather  than  successively 

To  Manage 
Intrinsic 
Cognitive 
Processing 

Segmenting 

Pre-training 

Presenting  narrated  animation  in  learner-paced  segments 

Providing  pre-training  in  vocabulary  and  key  concepts  (e.g., 
outlines,  key  learning  objectives,  bottom  line  up  front) 

Modality 

Combining  animation  (visual)  with  narration  (auditory),  not 
animation  (visual)  with  text  (visual) 
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Table  5  (Continued) 

Features  of  Well-Designed  IMI  from  Mayer  and  Colleagues 


Goal 

Features 

Description 

Guided  Activity 

Prompting  learners  to  select,  organize,  and  integrate  new 
information 

Reflection 

Encouraging  self-reflection  to  activate  organization  and  integration 
of  new  information 

Feedback 

Providing  learners  with  proper  schemas  to  repair  misconceptions 

Worked  Examples 

Leveraging  worked  examples  to  show  how  to  work  though 
tasks/problems  step-by-step 

To  Encourage 
Generative 

Personalization 

Communicating  in  an  informal/conversational  style 

Processing 

Voice 

Narrating  in  a  non-accented  voice  rather  than  a  machine-simulated 
voice 

Pacing 

Allowing  learners  to  control  their  pace,  and  process  smaller 
chunks  of  information  in  working  memory 

Sequencing 

Ordering  information  to  move  from  old  (familiar)  information  to 
new  (unfamiliar)  information 

Clear  Structure 

Using  a  familiar  structure/pattern  for  presenting  information  (e.g., 
compare -contrast,  classification,  enumeration,  cause-effect) 

Table  5  also  describes  design  features  which  are  focused  on  three  different  goals  to 
influence  cognitive  processing.  The  first  group  of  features,  which  are  focused  on  extrinsic 
cognitive  processing,  seek  to  reduce  distractions  and  make  critical  infonnation  stand  out  to 
learners.  Applying  these  features  likely  helps  learners  better  identify  and  leam  relevant  content. 
These  techniques  are  focused  on  selecting  the  right  words  and  images,  making  sure  critical  ideas 
stand  out,  not  overloading  the  learner,  and  following  a  logical  use  of  visual  space  and  training 
time  in  the  design.  We  applied  these  techniques  by  limiting  the  amount  of  text  on  slides, 
selecting  images  to  support  the  key  learning  points,  and  limiting  each  span  of  narration  to 
approximately  20  seconds  to  not  overload  learners  and  help  maintain  their  attention  (see  Clark  & 
Mayer,  2008). 

The  second  set  of  features,  concerning  intrinsic  cognitive  processing,  focus  on  helping 
the  learner  to  make  sense  of  the  infonnation  they  are  learning.  This  goal  focuses  on  related 
principles  that  allow  learners  to  pace  their  learning,  make  use  of  preexisting  knowledge,  and 
reduce  attention  conflicts  within  single  sensory  modalities  (e.g.,  presenting  static  text  with 
animation).  We  applied  these  techniques  by  providing  a  branching  navigation  system  within  the 
IMI,  providing  checks  on  learning,  and  for  the  tailored  training  IMI,  pre-  and  post-diagnostic 
assessments. 
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The  last  set  of  features  concern  generative  processing  and  are  intended  to  support  learners 
in  achieving  a  deeper  understanding  of  the  material  they  are  learning.  In  terms  of  these  features, 
the  goal  is  to  reduce  as  much  as  possible  the  barriers  to  understanding  by  establishing  an 
appropriate  communication  style  with  a  learner  (Clark  &  Mayer,  2008).  In  order  to  accomplish 
this,  we  used  tenninology  familiar  to  our  intended  military  learners  in  order  to  introduce  new 
concepts  and  made  connections  between  old  (familiar)  and  new  (unfamiliar)  infonnation. 

In  our  initial  review  of  existing  Anny  IMI,  we  found  both  positive  and  negative  examples 
of  design  features  to  consider  (see  Blankenbeckler,  Graves,  &  Wampler,  2013).  For  example,  a 
single  path,  “three  to  five  frames  and  a  check  on  learning  quiz”  approach  was  taken  in  some 
lessons  we  reviewed.  This  seemed  to  be  a  cost-efficient,  but  less  effective  approach  for  our 
needs.  While  it  was  perhaps  suitable  for  a  general  training  audience  or  to  introduce  the  task  to  an 
inexperienced  population,  it  has  a  very  low  level  of  interactivity.  Simply  reviewing  selected 
sections  of  a  field  manual  might  provide  an  equally  effective  learning  experience. 

Likewise,  we  reviewed  some  examples  of  high-end  multimedia  IMI,  which  relied  on 
extensive  video  with  staged  action  scenes  and  a  decision-action-consequence  structure.  While  it 
seemed  effective  and  engaging,  this  design  approach  is  very  costly  and  time  consuming  to 
produce.  Moreover,  this  type  of  IMI  would  require  a  very  high  bandwidth  connection  to  be 
deployed  online.  While  certainly  engaging  and  often  entertaining,  these  products  would  not  lend 
themselves  to  adaptation  or  quick  review. 

Our  designs  and  content  ultimately  incorporated  the  same  standards  we  used  to  select 
materials  for  reuse,  by  reverse  engineering  the  criteria  that  was  being  used  to  evaluate  existing 
IMI  for  reuse.  Table  6  provides  a  summary  of  these  considerations,  which  emerged  during  the 
previous  phase  of  this  effort,  as  detailed  in  Blankenbeckler,  Graves,  and  Wampler  (2013). 

Table  6 

Point  of  Need  Design  Criteria:  Reverse  Engineering  of  the  Criteria  for  Evaluating  Existing 


IMI 


Criterion 

Point  of  Need  Design  Considerations 

Complexity/Depth  of 
Information  Presented 

Provide  sufficient  content  for  the  desired  coverage  of  the  task/subject,  but  enable 
down-select  to  focus  on  a  specific  learning  need. 

Viable  Examples 

Provide  contextually  relevant  examples.  Use  multiple  or  varied  examples  that 
permit  Soldiers  to  generalize  principles  by  deriving  consistent  patterns  across 
examples.  Worked  examples  of  correct  procedures  would  be  used,  as  well  as  the 
detection  of  the  errors  of  others.  Apply  tailoring  principles:  backward  fading, 
withdrawal  of  scaffolding,  etc.  when  appropriate. 

Narrative  Flow 

Provide  a  coherent,  logical  narrative.  Assure  that  pages/frames  that  follow  build 
on  preceding  frames.  Integrate  tactical  examples  or  situations  when  possible. 

Presentation  is  Focused  vs. 
Diffused 

Maintain  a  clear  topical  focus  and  do  not  permit  training  to  meander. 
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Table  6  (Continued) 

Point  of  Need  Design  Criteria:  Reverse  Engineering  of  the  Criteria  for  Evaluating  Existing 
IMI 


Criterion  Point  of  Need  Design  Considerations 

Outcome  Meets  Goal  Focus  training  to  meet  the  goal.  Address  the  specific  point  of  need  and  target 

audience. 


Grouping  of  Content  Group  information  and  blocks  in  a  logical  coherent  structure. 

Timing  Present  the  material  in  a  manner  that  permits  the  learner  to  understand  a  learning 

point  before  advancing  to  the  next.  However,  structure  should  facilitate  the 
learner  jumping  around  in  the  media  to  quickly  review  relevant  points,  if  needed. 


In  addition  to  the  criteria  summarized  in  Table  6,  we  recognized  the  targeted  nature  of 
point  of  need  training.  Each  module  would  need  to  be  custom-made,  designed  to:  (a)  fit  the 
unique  characteristics  of  the  task,  (b)  cover  the  topic  to  the  desired  level  of  familiarization,  core, 
or  tailored  training  IMIs,  and  (c)  engage  the  targeted  training  audience.  Moreover,  each  of  the 
criteria  we  derived  in  our  initial  review  mapped  onto  the  IMI  design  goals  described  in  Mayer 
and  his  colleagues’  work  (i.e.,  Clark  &  Mayer,  2008;  review  Table  5).  For  instance,  timing  and 
grouping  of  content  relate  to  the  goal  of  reducing  extraneous  processing.  Providing  viable 
examples,  focusing  the  presentation,  and  carefully  attending  to  the  complexity  of  the  information 
presented  relate  to  managing  intrinsic  cognitive  processing.  Some  criteria,  such  as  narrative 
flow,  cross-cut  many  of  Mayer’s  goals  to  include  the  goal  of  encouraging  generative  processing. 
A  table  including  all  criteria  we  used  is  included  in  Appendix  A. 

While  these  features  are  important  for  designing  effective  IMI,  training  developers  and 
educators  should  also  consider  how  the  different  design  features  combine  to  best  fit  different 
learners.  Different  learners  may  benefit  most  from  different  types  of  instructional  designs,  with 
more  advanced  learners  potentially  benefitting  more  from  less  structured  designs  than  learners 
who  are  new  to  a  knowledge  domain  (Dyer,  Singh,  &  Clark,  2005).  In  other  words,  given  that 
learners  have  different  background  knowledge  and  learning  needs,  IMI  can  be  designed  to 
specifically  address  those  needs  in  terms  of  its  content,  features,  and  design. 

Conclusion 

This  research  report  is  the  second  in  a  series  of  three  related  to  our  research  on 
incorporating  the  ALM  point  of  need  concept  into  Army  IMI.  The  first  report  presented  the 
results  of  a  survey  of  existing  Army  IMI  focused  on  Combat  Arms  (see  Blankenbeckler,  Graves, 
&  Wampler,  2013).  This  report  presents  our  process  and  rationale  in  developing  six  IMI 
exemplars  focused  on  three  types  of  learning  needs:  familiarization,  core,  and  tailored  training.7 
The  third  report  will  present  the  findings  of  an  experimental  test  of  the  point  of  need  IMI  we 
conducted  with  NCOs  enrolled  in  the  Warrior  Leader  Course. 


7  Please  contact  Dr.  Rhett  Graves  at  the  U.S.  Army  Research  Institute  for  copies  of  the  IMI  exemplars: 
thomas.r.graves5.civ@mail.mil. 
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Our  main  focus  in  this  report  was  on  instructional  design.  Instructional  design  can  reflect 
the  ways  in  which  infonnation  is  transformed,  packaged,  and  then  communicated  for  an  audience 
of  learners  (see  Swanson  et  ah,  2013).  The  question  for  point  of  need  training  is  how  to  best 
transform,  package,  and  communicate  infonnation  to  address  the  needs  of  a  specific  audience. 
The  selected  technology  to  deliver  training  is  an  important  part  of  the  equation,  but  it  is  a 
different  type  of  question  than  that  asked  about  instructional  design.  The  answer  to  instructional 
design  questions  are  related  to  the  cognitive,  rather  than  the  geographic  accessibility  of 
information — i.e.,  the  training  materials  are  appropriately  complex  to  be  engaging, 
understandable,  and  effective  (Harun,  2002;  Walsh,  2010). 

Effective  instructional  design  also  addresses  the  recognized  learning  needs  of  an 
audience.  The  audience  often  shares  common  characteristics  in  terms  of  its  level  of  knowledge 
and  experience.  Shared  understandings  and  experiences  may  influence  the  way  the  respective 
groups  learn.  For  instance,  Dyer,  Singh,  and  Clark  (2005)  found  that  highly  structured  training 
was  beneficial  for  Soldiers  in  One  Station  Unit  Training  (OSUT),  whereas  Soldiers  enrolled  in 
the  Infantry  Officer  Basic  Course  (IOBC)  benefitted  from  a  less  structured  training  environment. 
For  OSUT  Soldiers,  training  materials  controlled  what  and  how  infonnation  was  being 
presented;  for  IOBC  Soldiers,  learners  were  allowed  to  self-select  how  they  were  accessing 
infonnation.  Further,  these  two  groups  of  Soldiers  were  different  in  average  age  and  civilian 
education  level.  Yet,  each  group  appeared  to  benefit  more  from  a  training  design  that  fit  their 
respective  level  of  knowledge  and  experience.  In  addition  to  understanding  what  makes  a  group 
of  learners  unique,  it  is  also  important  to  understand  the  impact  that  differences  among 
individuals  can  have  on  learning.  Any  IMI  that  can  address  the  challenge  presented  by  AFM’s 
point  of  need  concept  requires  an  instructional  design  to  address  defined  learning  needs  of  a 
particular  audience  and  individual  learners  (see  Pashler,  McDaniel,  Rohrer,  &  Bjork,  2009). 

This  research  report  focused  on  the  process  and  rationale  in  developing  six  IMI 
exemplars  designed  to  address  the  AFM’s  point  of  need  concept.  Point  of  need  is  intended  to 
both  address  the  geographic  accessibility  of  information  by  way  of  technology,  and  the  cognitive 
accessibility  of  information  by  way  of  instructional  design  (Blankenbeckler,  Graves,  & 

Wampler,  2013). 

To  develop  the  IMI  exemplars,  we  reviewed  existing  Anny  IMI  for  potential  reuse  in 
developing  new  point  of  need  IMI  focusing  on  three  types  of  Soldiers’  learning  needs: 
familiarization,  core,  and  tailored  training.  Most  existing  materials  were  designed  using  a 
different  pedagogic  approach,  addressing  a  wide  audience,  rather  than  addressing  more  narrowly 
defined  learning  needs.  Approximately  30%  of  the  material  used  in  our  exemplars  is  reused; 
the  remainder  needed  to  be  developed  anew. 

We  used  a  process  to  develop  the  IMI  that  involved  multi-phased  drafting,  evaluation, 
development,  and  re-evaluation  activities.  Storyboards  were  developed  and  reviewed  by  the 
research  team.  Once  storyboards  were  evaluated,  corrected,  and  revised,  they  were  passed  to  the 
graphic  design  and  programming  team.  Then,  the  graphic  design  and  programming  team 
produced  an  Alpha  version  of  the  IMI  module  for  review.  The  research  team  reviewed  and  made 
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corrections  to  the  Alpha  version;  the  corrections  were  sent  to  the  graphic  design  and 
programming  team  for  development  into  the  Beta  version  of  IMI. 

To  design  IMI  to  address  point  of  need,  designers  and  developers  should  focus  first  on 
instructional  design  issues  to  meet  an  identified  learning  need,  then  on  incorporating  novel 
technologies.  We  identified  and  applied  a  number  of  instructional  design  techniques  to  reduce 
extrinsic  processing,  manage  intrinsic  processing,  and  increase  generative  processing  (see  Table 
5;  also  Moreno  &  Mayer,  2007).  Many  of  these  techniques  were  supported  by  our  review  of 
existing  Anny  IMI,  which  enabled  us  to  identify  positive  and  negative  features  of  different 
instructional  designs  for  ALM  learner-centric  context. 

Large-scale  reuse  of  existing  materials  to  develop  point  of  need  IMI  had  a  low  payoff. 
The  availability  of  materials,  time  required  to  integrate  located  materials,  and  technical 
complexities  of  completing  the  integration  made  designing  new  IMI  a  more  efficient  and  far 
more  effective  option.  Transfonning  materials  made  for  the  “one  size  fits  all”  training  approach 
into  point  of  need  training  proved  a  difficult  challenge.  While  existing  IMI  did  provide  images 
and  a  few  animation  segments  of  value  to  our  targeted  point  of  need,  little  of  the  existing  IMI 
was  reusable.  The  search  for  those  reusable  fragments,  modifying  or  correcting  them  to  user 
acceptability  or  doctrinal  correctness,  and  integrating  them  into  the  instructional  design  proved 
more  time  consuming  than  creating  them  anew.  While  LCMSs  could  potentially  simplify  this 
process  in  the  future,  the  larger  the  anticipated  data  base,  the  more  disciplined,  regimented,  and 
restricted  it  would  need  to  be.  The  availability  of  related  materials  is  not  a  direct  indication  that 
they  can  be  integrated  into  an  instructional  design  that  meets  the  needs  of  the  target  training 
audience. 

Communicating  design  intent  and  reuse  parameters  to  programmers  can  be  done  with 
effective  storyboards.  Even  in  the  absence  of  source  materials,  technically  savvy  software 
technicians  were  able  to  frame  workarounds  that  captured  and  integrated  the  desired  materials. 
However,  care  must  be  exercised.  The  storyboard  cannot  become  so  filled  with  corrections  and 
changes  that  the  intent,  teaching  point,  or  desired  activity  or  interactivity  becomes  confusing. 
Instructional  intent  must  be  clear  to  avoid  delaying  the  production  process  or  causing  extensive 
edits  and  changes  to  materials. 

The  two  factors  that  proved  most  critical  to  our  effort  were  (1)  defining  the  needs  of  the 
target  population,  and  (2)  selecting  an  instructional  design  approach  that  addresses  the  needs  of 
that  population.  One  of  the  primary  reasons  that  most  IMI  is  unsuitable  for  reuse  is  that  it  is 
initially  designed  to  address  a  broad  audience  and  not  focused  on  a  point  of  need  audience. 
Well-designed  point  of  need  training  should  not  only  address  what  to  do,  but  why  and  when  to 
use  the  knowledge  and  skills  trained.  Successful  determination  of  the  needs  of  the  target 
audience  and  tailoring  the  materials  to  specifically  address  those  needs  in  terms  of  design, 
content,  and  features  provides  the  most  effective  learning  experience  for  an  audience. 
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Recommendations 


In  summary,  we  offer  the  following  recommendations  to  any  instructional  designers 
interested  in  adapting  existing  IMI,  or  creating  new  IMI,  focused  on  specific  audiences’  learning 
needs: 

•  Specify  the  needs  of  learners  in  terms  of  their  point  in  career,  prior  knowledge,  and 
experience  in  order  to  select  appropriate  content. 

•  Maintain  focus  on  the  type  of  technology  you  are  designing  the  instruction  for,  i.e., 
porting  IMI  designed  for  a  computer  screen  to  a  hand-held  device  will  likely  cause 
problems  with  readability,  etc. 

•  Remember  that  low-tech  and  well-designed  IMI  is  more  effective  than  poorly  designed 
IMI  with  lots  of  “bells  and  whistles”. 

•  Design  instruction  to  reflect  a  coherent  pedagogy,  i.e.,  design  problem-based  instruction 
around  actual  problems  from  the  field. 

•  If  existing  IMI  is  to  be  reused,  ensure  that  all  source  files  are  available  and  can  be  edited 
in  current  versions  of  software.  As  a  workaround,  sections  of  existing  IMI  can  be 
recorded  while  playing  within  the  host  LMS.  The  new  audio  and  video  files  can  be 
edited  to  fit  the  new  instructional  design  objective,  avoiding  the  unintended  problems  that 
arise  when  making  modifications  at  the  level  of  source  code.  The  older  the  source  files 
are,  the  more  problems  that  may  arise.  The  Anny  needs  a  centralized  repository  for  all 
IMI  source  files.  This  would  be  a  tremendous  asset  to  future  efforts  to  tailor  IMI  to 
Soldiers’  point  of  need  and  is  needed  if  units  across  the  Army  plan  to  exploit  existing 
IMI. 

In  terms  of  specific  design  features  to  incorporate  into  needs-focused  IMI,  we 
recommend  the  following: 

•  Whenever  possible,  frame  assessments  and  checks-on-leaming  around  tactical  scenarios 
in  order  to  reinforce  contextual  understanding  and  familiarity  with  domain-specific 
problems.  Avoid  framing  questions  and  checks-on-leaming  around  trivial  content;  in 
other  words,  reinforce  concepts  and  processes  not  acronyms. 

•  Keep  learners  moving;  as  often  as  possible,  limit  the  amount  of  text,  graphics,  and 
narration  on  a  single  frame  to  require  about  20-30  seconds  of  focused  attention  before 
moving  to  the  next  frame.  This  helps  to  maintain  learners’  interest  and  motivation. 

•  Limit  on-screen  text;  we  sought  to  use  no  more  than  a  third  of  a  page  if  text  was 
necessary  to  support  the  presentation. 

•  Select  images  to  serve  a  specific  purpose,  i.e.,  Soldiers  engaged  in  actions  related  to  the 
teaching  point,  and  use  commonly  understood  military  symbols  and  graphics.  We  found 
VBS2  to  be  very  effective  to  create  graphical  images  and  movies  to  support  instructional 
points. 

•  Use  legends  or  explanations  to  support  more  obscure  selections  of  symbols  and  graphics. 

•  Organize  the  IMI  content  into  brief  encapsulated  “chunks,”  and  provide  an  overall 
structure  content  (e.g.  hyperlinks)  that  learners  can  use  to  navigate  (i.e.,  provide  an 
advance  organizer).  Give  learners  control  over  how  they  move  through  the  IMI. 

•  When  progressing  through  IMI,  take  advantage  of  prior  knowledge  and  skills.  Use  old, 
familiar  concepts  and  terminology  to  introduce  new,  unfamiliar  infonnation. 
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•  For  sequential  tasks,  part-task  assessment  and  feedback  can  be  helpful  to  establish  an 
understanding  of  the  process  being  trained  and  the  rationale  for  each  step. 

•  Set  task-based  assessments  in  a  specific  context  and  relate  them  to  real  situations  the 
learner  may  encounter  (e.g.,  diagnosing  and  correcting  mistakes  in  a  previous  leader’s 
established  defensive  positions). 

•  Provide  diagnostic  assessment  and  feedback  before  training  to  assist  learners  in  selecting 
appropriate  content  to  maximize  their  learning,  and  after  training  to  help  learners  plan  for 
additional  learning. 

•  Provide  easy  access  to  additional  learning  resources  such  as  field  manuals  and  technical 
manuals. 
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Acronyms 


ACU 

AKO 

ALM 

ARI 

Army  Combat  Uniform 

Army  Knowledge  Online 

Anny  Learning  Model 

Anny  Research  Institute  for  the  Behavioral  and  Social  Sciences 

BDU 

Battle  Dress  Uniform 

CCM 

CSS 

Close  Combat  Missile 

Cascading  Style  Sheets 

DOTD 

Directorate  of  Training  and  Doctrine 

ELO 

Enabling  Learning  Objective 

FDC 

FCoE 

FIST 

Fire  Direction  Center 

Fires  Center  of  Excellence 

Fire  Support  Team 

HTMF 

Hypertext  Markup  Language 

IMDP 

IMI 

IOBC 

Instructional  Media  Design  Package 

Interactive  Multimedia  Instruction 

Infantry  Officer  Basic  Course 

FCMS 

FMS 

Learning  Content  Management  System 

Learning  Management  System 

MCoE 

MT2 

Maneuver  Center  of  Excellence 

MyTraining  Tab 

OSUT 

One  Station  Unit  Training 

SCO 

SCORM 

SGT 

SSG 

Sharable  Content  Object 

Sharable  Content  Object  Reference  Model 

Sergeant 

Staff  Sergeant 

TEO 

TLP 

TRADOC 

Terminal  Learning  Objective 

Troop  Leader  Procedures 

U.S.  Army  Training  and  Doctrine  Command 

VBS2 

Virtual  Battle  Space  2 

WLC 

Warrior  Leader  Course 
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APPENDIX  A 

INITIAL  CRITERIA  FOR  EVALUATING  EXISTING  IMI 


A-l 


Initial  Criteria  for  Evaluating  Existing  IMI 


Criterion 

Description 

Complexity/Depth  of  Information 
Presented 

Is  there  sufficient  content  to  provide  the  desired  coverage  of  the  subject  area, 
especially  when  attempting  to  fonn  the  different  points  of  need  training  material? 

Formal/Doctrinal  Correctness 
(Experts’  Perspective) 

Is  there  anything  that  would  raise  subject  matter  experts’  and/or  trainers’  concerns 
with  the  currency  and  accuracy  of  the  material?  Are  there  potential  negative  transfer 
issues? 

Repurposable  Graphics/Images 

Are  there  alternative  principles/ideas  demonstrated  by  the  same  image/graphic?  Are 
the  graphics/images  contextualized  to  guide  how  a  Soldier  is  interpreting  them? 

Verisimilitude/Face  Validity 
(Learners’  Perspective) 

Would  Soldiers  see  the  information  and  images  presented  as  being  accurate 
representations  of  current  Army  knowledge  and  practice  (e.g.,  current  doctrine,  up- 
to-date  uniforms,  and  current  weapon  systems)? 

Viable  Examples 

How  are  examples  being  used  and  are  there  single  or  multiple  examples?  Multiple 
examples  may  be  better  to  allow  Soldiers  to  generalize  principles  by  deriving 
consistent  patterns  across  examples  (cf.  Schwartz  &  Bransford,  1998). 

Narrative  Flow 

Does  the  narrative  of  the  training  make  sense?  Do  instructional  pages  that  come  later 
logically  build  on  what  came  before? 

Presentation  is  Focused  vs.  Diffuse 

Do  the  parts  have  a  clear  topical  focus,  or  does  the  training  meander? 

Outcome  Meets  Goal 

Does  the  training  reasonably  appear  to  accomplish  established  goals? 

Grouping  of  Content 

Are  the  modules  and  information  grouped  in  a  way  that  makes  sense  and  provides  a 
coherent  structure  (i.e.,  support  development  of  schemas)? 

Appropriate  Testing 

Do  the  tests  legitimately  cover  the  material  at  a  conceptual  level,  or  are  they  focused 
on  insignificant  details  and  off  the  subject  area? 

Interactivity/ C  ontrol 

In  what  ways  is  the  Soldier  being  asked  to  interact  with  the  training  package, 
materials,  etc.?  Is  the  interactivity  distracting  to  or  supportive  of  the  overall  goals  of 
the  training?  Is  the  Soldier  given  a  sense  of  being  able  to  shape  his/her  own  learning 
process? 

Timing 

Would  a  Soldier  be  able  to  develop  understanding  of  one  learning  point  before  the 
next  one  is  presented?  Are  there  logical  points  at  which  Soldiers  can  take  a  break 
from  what  they  are  learning? 

Use  of  Prior  Knowledge 

Is  prior  knowledge  elicited  or  refreshed  before  it  is  built  upon? 

Technical  Characteristics 

Are  the  requisite  files  available  and  in  a  fonnat  to  be  able  to  pull  apart  the  training 
package?  Is  the  software  package  that  was  used  in  developing  the  IMI  courseware 
current  and  useable?  Are  the  respective  pieces  of  the  courseware  files  able  to  be 
reconfigured  within  more  current  software? 

Suitability  for  Tailoring 

Does  the  courseware  include  aspects  that  would  support  various  tailoring  techniques 
(e.g.,  prompts  based  on  thematic  model,  structuring  presentation  of  elaborated/basic 
vs.  advanced  material,  color  cuing,  pretesting  and  modifying  learning  presentation 
based  on  performance) 

A-2 


